{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# XGBoost, LightGBM and CatBoost Parameter Tuning: Results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using a GPU, we can train a model in a few minutes and evaluate several hundred parameter combinations in a matter of hours, which would take many days using the sklearn implementation. \n",
    "\n",
    "For the LightGBM model, we explore both a factor version that uses the libraries' ability to handle categorical variables and a dummy version that uses one-hot encoding."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The evaluation relies on the results produced by the notebook [xgboost_lightgbm_catboost_tuning](05_xgboost_lightgbm_catgboost_tuning.ipynb) and stored in the `results/model_tuning.h5` HDF5 store. Sample results for the below outputs shown in the book are included. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports & Settings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-24T05:01:42.541619Z",
     "start_time": "2018-10-24T05:01:42.432927Z"
    }
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import warnings\n",
    "from pathlib import Path\n",
    "import os\n",
    "from datetime import datetime\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import graphviz\n",
    "\n",
    "from statsmodels.api import OLS, add_constant\n",
    "from sklearn.tree import DecisionTreeRegressor, export_graphviz\n",
    "from sklearn.externals import joblib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-24T05:01:45.749993Z",
     "start_time": "2018-10-24T05:01:45.747432Z"
    }
   },
   "outputs": [],
   "source": [
    "warnings.filterwarnings('ignore')\n",
    "sns.set_style(\"whitegrid\")\n",
    "np.random.seed(42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-24T05:01:46.118739Z",
     "start_time": "2018-10-24T05:01:45.932882Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.io.pytables.HDFStore'>\n",
      "File path: results/model_tuning.h5\n",
      "/catboost/factors/results             frame        (shape->[96,64])   \n",
      "/holdout/features                     frame        (shape->[16695,84])\n",
      "/holdout/target                       series       (shape->[16695])   \n",
      "/lightgbm/dummies/results             frame        (shape->[396,29])  \n",
      "/lightgbm/factors/results             frame        (shape->[396,29])  \n",
      "/lightgbm/holdout/features            frame        (shape->[16695,84])\n",
      "/lightgbm/holdout/target              series       (shape->[16695])   \n",
      "/xgboost/dummies/results              frame        (shape->[216,20])  \n",
      "/xgboost/holdout/features             frame        (shape->[16695,84])\n",
      "/xgboost/holdout/target               series       (shape->[16695])   \n"
     ]
    }
   ],
   "source": [
    "with pd.HDFStore('results/model_tuning.h5') as store:\n",
    "    print(store.info())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Keep only parameters used for grid search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-24T16:47:56.744556Z",
     "start_time": "2018-10-24T16:47:56.594337Z"
    },
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/xgboost/dummies/results\n",
      "/lightgbm/factors/results\n",
      "/lightgbm/dummies/results\n",
      "/catboost/factors/results\n"
     ]
    }
   ],
   "source": [
    "with pd.HDFStore('results/model_tuning.h5') as source:\n",
    "    with pd.HDFStore('results.h5') as target:\n",
    "        results = [k for k in source.keys() if k.endswith('results')]\n",
    "        for result in results:\n",
    "            print(result)\n",
    "            df = source[result]\n",
    "\n",
    "            # drop info that doesn't change across models\n",
    "            uniques = df.astype(str).nunique()\n",
    "            df = df.loc[:, uniques[uniques>1].index]\n",
    "            target.put('/'.join(result[1:].split('/')[:2]),\n",
    "                           df.sort_values('valid', ascending=False).reset_index(drop=True))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-24T16:47:57.244384Z",
     "start_time": "2018-10-24T16:47:57.208133Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.io.pytables.HDFStore'>\n",
      "File path: results.h5\n",
      "/catboost/factors            frame        (shape->[96,9]) \n",
      "/lightgbm/dummies            frame        (shape->[1,14]) \n",
      "/lightgbm/factors            frame        (shape->[1,14]) \n",
      "/xgboost/dummies             frame        (shape->[216,9])\n"
     ]
    }
   ],
   "source": [
    "with pd.HDFStore('results.h5') as store:\n",
    "    print(store.info())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Validation Performance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When comparing average cross-validation AUC across the four test runs with the three libraries, we find that CatBoost produces a slightly higher AUC score for the top-performing model, while also producing the widest dispersion of outcomes, as shown in the below graph."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### All models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-24T17:46:00.696786Z",
     "start_time": "2018-10-24T17:46:00.645201Z"
    },
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " xgboost\n",
      "      rounds     train     valid booster  learning_rate  gamma  max_depth  \\\n",
      "0  59.166667  0.798142  0.683618  gbtree            0.1      1          9   \n",
      "2  59.166667  0.798142  0.683618  gbtree            0.1      0          9   \n",
      "1  59.166667  0.798142  0.683618  gbtree            0.1      5          9   \n",
      "3  47.666667  0.790726  0.681356    dart            0.1      1          9   \n",
      "5  47.666667  0.790726  0.681356    dart            0.1      0          9   \n",
      "\n",
      "   colsample_bytree        time  \n",
      "0               1.0   85.372013  \n",
      "2               1.0   75.323257  \n",
      "1               1.0   94.299514  \n",
      "3               0.8  564.300030  \n",
      "5               0.8  490.855656  \n"
     ]
    }
   ],
   "source": [
    "with pd.HDFStore('results.h5') as store:\n",
    "    for key in store.keys()[:1]:\n",
    "        df = store[key].sort_values('valid', ascending=False)\n",
    "        print('\\n', key.split('/')[1])\n",
    "        print(df.head())\n",
    "        df.iloc[0].to_csv('catboost_best.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-24T16:57:26.188510Z",
     "start_time": "2018-10-24T16:57:26.151658Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 1104 entries, 0 to 215\n",
      "Data columns (total 4 columns):\n",
      "train             1104 non-null float64\n",
      "Validation AUC    1104 non-null float64\n",
      "time              1104 non-null float64\n",
      "Model             1104 non-null object\n",
      "dtypes: float64(3), object(1)\n",
      "memory usage: 43.1+ KB\n"
     ]
    }
   ],
   "source": [
    "models = ['catboost/factors', 'lightgbm/dummies', 'lightgbm/factors', 'xgboost/dummies']\n",
    "df = pd.DataFrame()\n",
    "with pd.HDFStore('results.h5') as store:\n",
    "    for k in models:\n",
    "        model = k.split('/')[0]\n",
    "        data = k.split('/')[1]\n",
    "        df = pd.concat([df, store[k][['train', 'valid', 'time']].assign(model=model, data=data)])\n",
    "df['Model'] = df.model.str.capitalize().str.cat(df.data.str.capitalize(), sep=' ')        \n",
    "df = df.drop(['model', 'data'], axis=1).rename(columns={'valid': 'Validation AUC'})\n",
    "df.info()    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-24T17:05:59.692441Z",
     "start_time": "2018-10-24T17:05:59.678584Z"
    }
   },
   "outputs": [],
   "source": [
    "top_scores = df.groupby('Model')['Validation AUC'].max().sort_values(ascending=False).to_dict()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-24T17:16:58.929694Z",
     "start_time": "2018-10-24T17:16:58.710584Z"
    },
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA+gAAAGoCAYAAADVZM+hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3XdUFNfbwPHvLlWKIgaJSrAQxUIRSyyIBaxYQLAjlliiMWKwEHsBS8QejSWW2GNFxBZRRMWaiA07Yi8oKiJFWLa8f/A6P1dQ0QBicj/n5ISduTPz3NlZd5+5ZWQajUaDIAiCIAiCIAiCIAiflPxTByAIgiAIgiAIgiAIgkjQBUEQBEEQBEEQBKFQEAm6IAiCIAiCIAiCIBQCIkEXBEEQBEEQBEEQhEJAJOiCIAiCIAiCIAiCUAiIBF0QBEEQBEEQBEEQCgGRoAuCIAifnK2tLba2tty9ezfbuj/++ANbW1vmzJnzCSLLEhcXx7Bhw3B2dsbJyYmOHTuyd+/eTxbP67p168aPP/6Y47rIyEjs7OxITEx85z6USiW2tracPHkSgIYNGxISEpJj2bi4OGxtbYmPj89VfMePHycuLg6AzZs34+rqmqvtPtTw4cOl68jW1pYqVapQv359xo4dS2pq6kfv98KFC7i6uuLo6MjRo0fzMOLC7+nTp3h4eKBQKLSWjx49GltbW65cuZJtm65duzJ//vxsy3O6btRqNatXr6Zdu3ZUr16dxo0bM2HCBJ49eyaVmT59Olu3bs3DWgmCIBRuIkEXBEEQCgU9PT0OHDiQbfn+/fuRyWSfIKIsZ86coWPHjhgaGrJkyRJCQ0Np3bo1Q4cOZdOmTZ8srlfatm3LoUOHyMjIyLZuz549NGjQgOLFi3/QPkNDQ3F3d//HsalUKnr16sWTJ0+kWLds2fKP9/s2rVq14siRIxw5coSDBw8yZ84cDh48yPTp0z96n/Pnz+frr79m165d1KxZMw+jLfxmzJhB9+7d0dfXl5YpFAr27dtH2bJl2bZt20fvW6PR4Ofnx8qVK/nuu+/YsWMHM2bM4PLly/Ts2ZOUlBQA+vfvz8KFC0lKSvrH9REEQfgciARdEARBKBRq1aqVLUFPSUnhzJkzVK1a9ZPEpNFoGDlyJC1btmTKlCnY2dlRtmxZevXqxcCBA5k1axYvX778JLG90rJlSzIzM4mKitJarlAoiIiIoF27dh+8T3NzcwwNDf9xbBqNRuu1oaEh5ubm/3i/b2NoaIiFhQUWFhZYWlpSp04dunXr9o96OyQnJ2NnZ4eVlVWenJPPxb1794iIiMDDw0Nr+eHDh9FoNPTo0YMdO3agVCo/av+hoaEcPnyYVatW0bp1a7766itq167Nb7/9xoMHD6SbX8WLF6devXps2LDhH9dJEAThcyASdEEQBKFQcHNzIzo6muTkZGnZoUOHqFWrFsbGxlpl9+/fT+vWrXF0dKR9+/YcPnxYWpeSksKYMWOoV68ednZ2tGjRQitBs7W1JTQ0lLZt2+Lk5ISvry937tzJMabTp09z69Yt+vTpk22dr68vv/32GwYGBpw8eZKGDRsSGBhIzZo1pS6+ISEhuLu74+DggJeXl9SFHODkyZN4eXnh4OBA48aNWbJkSa7Wval48eI0aNAgWxL6KpF61aU8JSWFUaNGSeelZcuW7N+/P8d9vt7FXaFQMHHiRGrWrEnjxo05duyYVtnr16/Tp08fnJycsLe3p1u3blKX9lfH7tGjBwsXLszWxT02NpY+ffpQo0YNXFxcWLBgAWq1GoA5c+YwbNgwJkyYQI0aNXB1dWXp0qVvPQ9vo6urq9UCvHfvXtzd3XF0dMTLy0ur23rXrl0JCgrCzc2Nhg0bUr16daKjo/n1119p1qwZAA8fPsTPz49vvvmGOnXqMHnyZKkL+ObNm+nSpQuDBg2iZs2abN26la5du7Js2TJ69uyJg4MDnTt35u7du4wePZrq1avTokULTp8+LcUQERGBp6cn9vb21KpVi6FDh0pd9N93TpRKJXPmzKFBgwbUqFGDQYMGkZCQAGR1J1+wYAENGjSgZs2a9OvX763XPcDGjRtp0KCB1rkD2LlzJ7Vq1cLV1ZWnT59qffY+REhICM2bN+err77SWm5mZsbvv/+Op6entMzV1ZUNGzZku+EjCILwbyQSdEEQBKFQsLGxoUyZMlo/+CMiImjatKlWuStXrjBixAj69evHjh076NSpEz/88AOXL18GYNq0acTFxbFixQp27txJ7dq1GTdunNY42gULFjB69GhWr17NkydPmD17do4xXblyBWNjY2xsbLKtK1asGI6OjsjlWV+ljx49IiUlhW3bttG+fXtCQkIIDAykf//+bN++HWdnZ/r378+DBw9QqVT4+fnRpEkTdu/ezfjx4/n111+Jiop657q3adOmDZGRkVp13LNnD82bN5dafSdPnszt27f5/fff2blzJ05OTowZMybb+OI3zZ07l6ioKJYsWcKsWbNYs2aNtE6tVjNgwACsra3Zvn07f/zxBwqFghkzZgBI3dnnz59Pr169tPb79OlTunfvTqlSpdi8eTPjx49nzZo1rF69Wiqzd+9ejIyM2Lp1K97e3sycOZMbN268M97XY7t48SJ//PGHdFPgwoULjBo1igEDBhAWFoa3tzcDBw7k2rVr0nYhISHMmDGDhQsXcuDAARwcHPj222/ZuHEjCoWCHj16oFAoWLNmDXPnzuXAgQMEBwdL2585c4avv/6aTZs20bhxYwAWLlxI165d2bJlC0+ePMHLywsrKyu2bt2KlZUV06ZNA+D27dsMGTIEHx8f9uzZw5w5czh69KjWUIp3nZN58+YREhLC1KlT2bx5M6mpqYwaNQqA1atXs3PnTmbOnMmmTZsoXbo0vXr1ynFoBGTd4Klfv77WstTUVCIjI3F1daV06dJUrVqV0NDQXL0fb7p69Sr29vY5rnNwcNDqaVGvXj3i4+O5fv36Rx1LEAThcyISdEEQBKHQcHV1lbq5Z2ZmcuTIkWyTii1fvhxvb288PT2xtrama9eutG7dWkoca9asyaRJk6hSpQrlypXj22+/JSkpiUePHkn76NmzJ/Xq1cPe3p6uXbsSExOTYzzJycmYmJjkOv6+fftibW2NlZUVa9aswcfHB09PT8qXL8+wYcOoXLkya9asITk5mefPn1OiRAmsrKxwdXVl5cqVVK5c+Z3r3sbNzQ2VSsXx48cByMjI4MCBA7Rt21YqU6tWLSZNmkTlypWl8/L8+XOePn361v2q1Wq2bt2Kn58ftWrVombNmgQEBEjr09LS6Ny5MwEBAVhbW2NnZ0f79u2lROpVklWsWDGMjIy09h0WFoaRkRETJ07ExsaGZs2aMXjwYJYtWyaVMTMzY8SIEZQvX55BgwZhYmLCxYsX3xrvjh07cHJywsnJCQcHBzp16kS1atUYNmwYACtWrKBTp060a9eOsmXL4uPjQ4sWLVi7dq20j8aNG1OjRg3s7OwwNzdHV1cXIyMjzM3NOXjwIE+ePCE4OBhbW1vq1avHuHHj+OOPP6RWbplMxsCBA7GxsaFEiRIANGnShJYtW1KpUiWaNGlCsWLF+P7777GxsaFDhw5Sgq1SqRg3bhwdO3bEysoKFxcX6tSpQ2xs7HvPiVqtZtOmTfj7+9OwYUNsbGyYOHEi1apVQ6PRsGzZMgICAqhbty42NjZMmDABjUbDvn37sp1HhULBtWvXst2Y2r9/PwqFQvpMNmvWjAMHDvD8+fO3vidvk5ycjKmpaa7KFilShNKlS7/1cyoIgvBvovupAxAEQRCEV9zc3Pj+++9RKpWcOHGCr7/+mi+++EKrTFxcHNeuXdOa2TkzMxMHBwcAPD092b9/P5s3b+bGjRtSQveq6zSAtbW19LeJiclbx9EWL15cq8v9+5QpU0YrzoEDB2qtr169Ojdu3MDMzIz+/fszadIkFi1aRJMmTWjXrh0WFhYAb1136tQp+vXrJ+3vu+++Y8CAARQpUgQ3Nzf+/PNPGjVqxMGDBzE2NqZu3bpSWS8vL8LDw9mwYYPWeVGpVG+tz5MnT3j+/LnWzQE7OzvpbxMTE7p27UpISAgXLlzg5s2bXLx4UarHu8TFxVGtWjV0df/3U8TJyYmEhARevHgBgJWVldRDAcDY2PidY55dXV0ZPnw4kNW1/YsvvsDAwEDrmOHh4WzcuFFalpmZiZOTk/T69fcwp5jLlStH0aJFtWJWKpVSd/ESJUpkG6tuZWUl/W1oaEjp0qW1Xr/qxVChQgUMDAxYtGgRsbGxxMbGcv36da1x4G87J6/eq2rVqknrypUrh7+/Py9evCAhIQF/f3+tbdPT07l161a2eiYmJqJWq7NNLrhz504cHR2lz2Tz5s2ZN28eu3btwsfHB8g6769/1l551T391fttZmYmvc+5YWZm9s6bSYIgCP8WIkEXBEEQCo0aNWqgo6NDdHQ0ERER0rjf16lUKvr06YOXl5fW8ldjZQMCAjh9+jQeHh507doVCwsLOnfurFVWT09P6/Xbxrba29uTlpZGbGwsFStW1Fr39OlThg8fzrhx46RlryeDOU0oplKppIR42LBhtG/fnoiICCIjI/H19WXy5Ml4e3u/dV3r1q21uhQXK1ZM+rtt27YEBASgVCrZs2cP7u7u6OjoSOuHDRvG+fPn8fDwoFu3bpibm9OtW7cc6/2m18/P6wl1SkoK3t7elChRAldXV9q1a8e1a9dYtWrVe/f5+rl6/fy8/v8336c3Y3mTsbExZcuWfet6lUpFv379tMY3vxlLTnG9a92rZPRVzDmVef2cAVpJ8usuXryIj48Pbm5u1KpVi969e7NixQqtMm87Jzktf+XVTY158+ZRvnx5rXWv32x45dVTE15PtBMTEzl27BgqlSrbpI3btm2TEvSiRYtKM7C/LikpCZlMJrWa29vbc/78+RzjXbBgAbq6ugwYMEBaplarta5nQRCEfyvRxV0QBEEoNORyOY0bN+bAgQNERkZmG38OUL58ee7evUvZsmWl/7Zv386+fftISUlh586dzJo1iyFDhtCsWTPp8UwfM8FU1apVqVSpUrYkCWDdunWcP3+eUqVK5bhthQoVOHfunNayc+fOUb58eRISEpg4cSJlypShX79+rF+/Hi8vL/bs2fPOdYaGhlr1NjMzk/bt7OyMXC7n+PHjHDp0SGv29qSkJHbv3s2cOXPw8/OjWbNmUrfkd52XL774guLFi2t1Lb506ZL094kTJ3jy5AmrV6+mb9++1KtXjwcPHkj7fNfj8SpUqMDFixe1WsTPnj2Lubm5Vr3yUvny5bl3757WOQwJCSEiIiJX29vY2HDr1i2tlt8zZ86gq6ur1SvjY4WGhvLNN98wa9YsunXrhr29Pbdv387VtVu8eHHMzMykuRggq8Xf2dkZPT09zMzMSEhIkOpdpkwZZs2apTX+/pUSJUogl8tJTEyUlu3ZsweNRsO6desIDQ2V/vv++++JiYmRJga0tbXlzJkz2fZ59uxZqYcAgIeHB/v27ePu3bta5RISElizZk22mxivhn0IgiD824kEXRAEQShU3Nzc2Lx5M2ZmZtlmeAbo1asXf/75JytXruT27dv88ccfLF68GGtrawwMDChSpAjh4eHcu3ePI0eOEBgYCPDeydDeZsKECezatYtx48Zx+fJl4uLimD9/PosXL2bs2LEUKVIkx+169+7N+vXrCQ0N5ebNm8yaNYvLly/TqVMnihUrxv79+5kyZQq3b9/m/PnznDp1imrVqr1z3bvo6urSqlUrZs2ahaWlpVZXdENDQ63zcvjwYaZMmfLe8yKXy+nWrRu//PILx48f5/z589IEcJDV7TgtLY39+/dz7949Nm7cKE0UB6Cjo4OBgQGxsbHZhgp4eHiQkZHBxIkTiYuLY//+/fz6669069Yt355736tXL3bv3s2qVau4c+cO69atY+nSpblOrhs0aICVlRUBAQFcvXqV48ePM3XqVNq2bZtjS/SHMjMz4+rVq5w/f56bN28yZcoULl68mOtr19fXl3nz5nH8+HGuX79OUFAQ9vb2mJqa0qtXL+bOnUtERAS3b99mwoQJnDx5kgoVKmTbj46ODra2tly9elVatmvXLlxcXKhZsyaVKlWS/uvZsycGBgbSM9E7dOhAXFwcQUFBXL9+nRs3brB582YWL16s9TQEd3d36tSpQ8+ePdmzZw93794lKiqKb7/9llKlStG9e3ep7IsXL3j48KHWNS0IgvBvJbq4C4IgCIVKgwYNUKvVObaeQ9Y47pkzZ7JgwQJmzpxJmTJlmDp1qjRj9owZM5g+fTrr1q3DysqKAQMGMH/+fC5dukSlSpU+OJ5atWqxdu1aFi5cyLfffkt6ejqVKlVi/vz5uLm5vXW7Fi1akJCQwC+//EJCQgJVqlTh999/l7rKL1q0iKlTp+Lp6YmBgQHu7u4MGjQIfX39t657n7Zt27Ju3Tr8/Py0lhsYGBAcHExwcDBr1qzBysqKQYMGMXv2bC5fvvzObuGDBg0iPT2dIUOGoKenx8CBAwkKCpLOzYABA5g0aRIZGRlUrlyZCRMmMGbMGB49eoSlpSW9evUiODiYBw8eaHWvNjExYdmyZUyZMgVPT09KlCjBt99+qzXGPq/VrFmTn3/+mYULFzJjxgysrKz4+eefcXFxydX2Ojo6LFy4kKCgIDp16oSxsTEeHh74+/vnSXw9e/bk6tWr9OzZE0NDQ2rXrs3333/P7t27c7X9d999R0pKCv7+/mRmZtKwYUNpCEb//v1JT09nwoQJJCcnU7VqVZYtW/bW+QIaNmxIdHQ0Pj4+xMfHEx0dLT0+8HVmZma4u7uzfft2/P39KV26NGvXrmXu3Ll069aNjIwMypUrx5gxY7SGFshkMhYuXMhvv/3G3LlziY+Px9zcHFdXV3744QetSQWjo6MpU6ZMjk9TEARB+LeRacRDJQVBEARBEITX3Llzh/bt2xMVFZVtBv6CNmzYMGxtbenfv/8njUMQBKEgiC7ugiAIgiAIghZra2saN25MWFjYJ43j6dOn/P3333Tp0uWTxiEIglBQRIIuCIIgCIIgZPPTTz+xbt26j56/IS8sWbKEwYMH58kYf0EQhM+B6OIuCIIgCIIgCIIgCIWAaEEXBEEQBEEQBEEQhEJAzOL+L3H27Fnp2aLCx8vIyBDnUShQ4poTCpq45oSCJq45oaCJa04oaLm55jIyMqhevfp79yUS9H8JAwMDqlSp8qnD+OxdvnxZnEehQIlrTiho4poTCpq45oSCJq45oaDl5pq7fPlyrvYlurgLgiAIgiAIgiAIQiEgEnRBEARBEARBEARBKAREgi4IgiAIgiAIgiAIhYAYg/4fp1aruXPnDklJSSiVyk8dzif38OFD0tLSPnUYQg4MDQ0pXbo0xYsX/9ShCIIgCIIgCEK+EAn6f1RGRgb79u3j8uXLmJmZUaJECXR1dZHJZJ86tE9KrVbz4MGDTx2G8Aa1Wk1GRga7d++mWLFi1KtXD3t7+08dliAIgiAIgiDkKZGg/wdlZGSwbt06zM3N6devH2ZmZp86JEHIFbVaza1btwgLCyM9PZ3atWt/6pAEQRAEQRAEIc+IMej/Qfv378fc3BwPDw+RnAufFblcToUKFejRoweHDh3i0aNHnzokQRAEQRAEQcgzIkH/j1Gr1Vy6dIlGjRr957uzC58vc3NzHBwcuHjx4qcORRAEQRAEQRDyjEjQ/2Pu3r1LsWLFxERbwmevatWqXLly5VOHIQiCIAiCIAh5RiTo/zFJSUmUKFHiU4chCP9YiRIlePHixacOQxAEQRAEQRDyjEjQ/2MyMzPR09P71GEIwj+mr69PZmbmpw5DEARBEARBEPKMmMVd0KJSqVi9ejU7duxApVKRmZlJkyZNGDJkCPr6+u/cdsGCBVSuXJmmTZsycuRIKlasSJ8+ffI13oMHD3Lu3DmGDBmSbZ2vry/379/H1NRUa/n27ds/6lhjx46lS5cu2NnZfdT2hcXevXtZsmQJCoWC0qVLM336dIoXL05qaiqjR48mLi4OtVqNt7d3ju9fly5dePnypfT65s2bdOrUibFjx3Lu3DkCAwNJS0ujZMmSzJgxg5IlS5KYmMjAgQN5+vQpnp6eDBo0CIBTp06xadMmgoODC6z+giAIgiAIglBYiRZ0QcvEiRM5c+YMq1atYvv27WzZsoWbN28yZsyY92578uRJlEplAUT5PzExMSQlJb11fUBAANu3b9f672MdO3YMjUbz0dsXBjExMQQFBfHLL7+wc+dOypUrx5w5cwBYsWIFhoaG7Ny5k02bNrF69WrOnz+fbR8bNmyQzqWfnx9WVlYMGTIEhUKBn58fo0ePZs+ePbRo0UK6bnbs2EGTJk0IDw9n9+7dpKSkoFKpmD17NsOHDy/QcyAIgiAIgiAIhZVoQRck9+7dY8eOHRw5cgQTExMAjIyMmDRpEqdPnwayWksDAwNJTU0lISGBypUrM3fuXLZs2cKFCxcIDg5GR0cHgOjoaPbu3UtKSgrOzs789NNP6OrqcurUKYKDg3n58iV6enr8+OOPNGzYEIBff/2VXbt2oaOjQ/ny5Rk3bhwWFhaEh4ezaNEiZDIZOjo6BAQEoK+vz4YNG1CpVJiamuLv75/rup49e5YZM2agUChISEigfv36TJ06FYDIyEjmzp2LWq2W6r9nzx4eP37M8OHDCQ4OxtLSkokTJ3L//n00Gg2enp707duXe/fu4ePjg42NDffv32flypUsXryY06dPo6enh5WVFdOmTcPY2Dgv37pcCwsLw9vbGysrKwAGDx7M8+fPgazeE6mpqSiVSjIyMlCr1e/sNfH8+XMmTJjAokWLMDU1JTo6GhMTE2rWrAlAhw4dmDp1KomJiejr6/P06VMyMjJQKpXI5XLWr1+Pm5sbJUuWzP+KC4IgCIIgCMJnQCToguTixYt8/fXXUnL+ioWFBS1atABg06ZNeHp64uHhQWZmJl5eXhw8eBAfHx/+/PNPfHx8aNasGREREcTHx7N27Vp0dXXp06cPmzZtolWrVvj5+bFo0SIcHR2JjY2le/fubNmyhb/++ouoqCi2bNmCkZER8+fPZ+TIkSxfvpzg4GBmzpxJ9erVOXLkCCdPnuSHH36gS5cuJCYmvjU5Dw4OZtGiRdLroUOH0qhRI1avXo2fnx916tQhNTUVNzc3Lly4wJdffsmIESNYvXo1VatWJTw8nJkzZ7Js2TJ27NjBzJkzsbe3p3v37ri5udG7d2+Sk5Px8fGhVKlSODo6Eh8fz6xZs6hVqxanTp3ir7/+Yvfu3chkMmbMmMHVq1epUaNG/r2R73Dr1i1sbW0ZOHAg9+/fp1KlSowePRqAvn374uvri4uLCykpKfj4+FC5cuW37mvp0qU0atQIe3t7AOLj4/nyyy+l9fr6+pibm/Po0SPatWtHQEAA3t7e9O7dm/T0dHbt2sWaNWvyt8KCIAiCIAiC8BkRCbogkcvlqNXqd5YZMWIER48eZenSpdy6dYvHjx+TlpaWY1kPDw+MjIwAaNeuHYcOHaJMmTJYW1vj6OgIQMWKFalRowZ//fUXhw8fxsvLS9qmR48eLF68GIVCQevWrfnhhx9o1KgRzs7O9OvXL1d1CggIoGXLltmW//zzzxw+fJjFixdz48YNMjIySEtL4/Tp01SsWJGqVasC0Lx5c5o3b6617atyK1asAMDU1BQvLy8OHz6Mo6Mjurq6VK9eHYBKlSqho6NDx44dadCgAS1atMDBwSFXsecHpVJJZGQkK1eupESJEsyYMYOxY8eycOFCAgMDcXZ2ZujQoTx58oTevXvj5OQk3Zx5XUZGBps2bSIkJERaplarkclkWuU0Gg06OjoYGRmxYMECafmYMWMYPHgwly5dYuHChejq6jJy5Ei++uqr/Ku8IAiCIAiCIBRyYgy6IHFwcODGjRukpKRoLX/06BH9+/cnPT2doUOHsmnTJsqUKUOvXr2oVq3aW8dlv+rqDlmJmq6uLiqVKsckTqlUZkvw1Gq1NKbd39+f9evXY2dnR0hICD4+Pv+ort27d+fQoUNUqFCBQYMGUbJkSSmZfD0GjUaT7VnbarU6W51fj1VfXx9d3ax7X0WLFmX79u389NNP6Ojo8OOPP7Ju3bp/FPs/UbJkSVxcXLCwsEAul+Pl5cXZs2cB2LdvH507d0Yul1OyZElatmzJyZMnc9zP4cOHqVy5slZCXapUKR4/fiy9zszM5Pnz51haWmpte/78eZKTk3F2dmbatGlMnDiR3r1788svv+RDjQVBEARBEATh8yESdEFiaWlJ27ZtGT16tJSkp6SkMHHiRMzMzDA0NOTIkSMMGjQId3d3AM6dO4dKpQKyEvLXJ4nbtWsXCoWCjIwMtm3bRsOGDalevTo3btyQJh+LjY3l77//5ptvvsHFxYWtW7dKLfJr1qyhdu3ayOVyXF1defnyJV27dmXChAlcvXoVhUKR7Zi58eLFC2JiYhg+fDjNmzcnPj6eO3fuoFarcXR0JC4ujtjYWAAiIiIYMWKEVv1MTExwdHSUEu3k5GRCQ0OpX79+tmNFRkbSq1cvnJycGDx4MJ6enly4cOGD4s1LLVq0IDIyksTERADCw8OlLupVq1Zlz549QFYvgaioKKmnw5v++usv6tWrp7XM0dGR58+fS/MVbN26lerVq1O0aFGpjEajITg4mICAAADpPZTL5aSnp+dtZQVBEARBEAThMyO6uAtaJkyYwMKFC+nSpQs6OjooFAqaNm3K4MGDgayW7EGDBmFkZISJiQm1a9fmzp07ALi6ujJ79mzp2dRWVlZ069aN1NRUmjVrRvv27ZHJZMybN4+goCDS09ORyWRMmzaN8uXLU7ZsWR4+fEjHjh1Rq9WULVuWmTNnoqury+jRoxk+fDi6urrIZDKmTp2Kvr4+devWZfjw4QQFBTFu3Lhc1bFo0aL079+f9u3bY2RkhKWlJTVq1OD27dvUq1ePmTNn8tNPP6FSqTAxMZFmOW/WrBkjRoxg4sSJzJw5k8DAQEJCQlAoFLRt2xYvLy/u37+vdayGDRty+PBh2rRpg5GREcWKFSMoKCj9OlQhAAAgAElEQVSv3q4P5urqSnx8PL6+vqjVakqXLs2UKVMAmD59OoGBgYSGhiKXy2nVqhUeHh5AVpd0Ozs7unbtCsDt27ezPW5OT0+PBQsWEBgYyMuXLzEzM2P69OlaZbZs2ULdunWlSeoGDhxI9+7d0dfXZ9q0afldfUEQBEEQBEEo1GSaz/25UQIAly9fpkqVKu8tFx0dzf3792nXrl0BRCUI+UelUjF16tRc35gprHL72RWEvCKuOaGgiWtOKGjimhMKWm6uudxel6KLuyAIgiAIgiAIgiAUAiJBFwRBEARBEARBEIRCQCTogiAIgiAIgiAIglAIiARdkNy7dw8nJ6cc182bN4/Q0NB3bh8SEsJ3332X47qDBw8yb96898bg6+vLn3/++f5gc2nkyJG4uLjg4eGBh4cH7u7ujB8/noSEhDw7Rm5FREQwefLkAj/um/bu3YuXlxdt2rShf//+0ozuqampDBkyhDZt2uDu7s7y5ctz3D45ORl7e3vpnHp4eHDixAmuX7+utaxt27bY2toSHh6OQqGgX79+NGvWjPHjx0v7unPnDr169SqIaguCIAiCIAhCoSdmcRdyZciQIf9o+5iYGJKSkvIomg/Tq1cv+vTpA2Q95mvJkiX07duXkJAQrWe15zc3Nzfc3NwK7Hg5iYmJISgoiA0bNmBlZcXUqVOZM2cOgYGBrFixAkNDQ3bu3ElKSgqtW7emdu3aODg4aO3j7Nmz1K5dmxUrVmTb//bt26W/f/75ZypVqkTz5s2JiIigVKlSLF26lD59+nDt2jUqVarEtGnTGDlyZL7XWxAEQRAEQRA+ByJBF3Jl5MiRVKxYkT59+nDo0CFmzpyJXC6nSpUqHDt2jPXr1wOQkJBA//79efjwITo6OsyaNYuUlBQ2bNiASqXC1NQUPz8/goODOXDgAKampjg4OBAXF8eaNWsA2LdvH7/99hvp6em0bduWgQMHcu/ePXr27ImzszMXLlxApVLh5+fHxo0buXHjBnZ2dsyePRu5/N2dQmQyGQMGDGDbtm0cPXqUChUq0LZtW86cOQNk9SJ49TokJITw8HDUajUPHjzA0tKSTp06sXbtWm7dukXv3r359ttvP6jc3r17WbJkCcnJyUyZMoVr166RmZlJvXr1CAgIQFdXl19++YV9+/ahp6dH8eLFmTZtGiVLlsyT9zEsLAxvb2/pMWeDBw/m+fPnQNas6KmpqSiVSjIyMlCr1ejr62fbx5kzZ3j+/DmdOnVCoVDQqVMnunXrplXm1KlT7N27lx07dgCgr69PWloaCoWC9PR09PT0iIyMpFSpUlSuXDlP6iYIgiAIgiAInzvRxV34IImJiQQEBDBjxgy2b99OnTp1ePTokbT+7t27jBkzhh07dlCrVi2WL1+Oo6MjXbp0wd3dHX9/fzZv3szFixfZuXMnGzZs4O7du1rHSE1NZdOmTWzatImwsDAOHToEZCXPjRo1IiQkhOrVqzNlyhRmz57Nrl27OHXqFGfPns11PWxtbbl27dp7y506dYpJkyYRFhZGfHw8u3btYtWqVSxdupS5c+eiVqs/qNwrU6dOpVq1aoSEhBAaGkpiYiK///47Dx8+ZNWqVWzdupWQkBCcnZ05f/58ruv1Prdu3UKlUjFw4EDatWvHpEmTMDY2BqBv377cv38fFxcXGjduTOvWrXNMnnV0dHB1dWXt2rUsWbKEVatWsX//fq0ywcHB/Pjjj5iYmADg7OyMgYEBnp6e1KlThzJlyrBo0aJ/3DNDEARBEARBEP5NRIIufJBTp05hY2MjJW7t27eXkjAABwcHypYtC0CVKlV49uxZtn0cOnQIDw8PDAwM0NfXp3PnzlrrO3TogK6uLiYmJrRo0YJjx44BoKenh6urKwDW1tY4OTlhYmKCgYEBJUuW/KAu9DKZjCJFiry3nL29PaVKlUIul2NlZUWDBg2Qy+V89dVXZGRk8PLlyw8q98rBgwfZuHEjHh4eeHl5cf78ea5du4alpSWVK1emffv2TJ8+nSpVqtC0adNc1+t9lEolkZGRBAYGEhoaioWFBWPHjgUgMDAQZ2dnjh49SkREBFFRUezduzfbPgYNGsQPP/yAvr4+lpaWdO7cmX379knrT58+zbNnz2jbtq20TC6XM2XKFHbv3o2fnx9Lly6lQ4cOJCYmMmjQIL777jsuXbqUZ/UUBEEQBEEQhM+R6OIufBAdHR00Go3Wste7levq/u+Skslk2cq+WebN7V8d4xWNRiOV19PTQyaTSev09PQ+ogZZ+7x48SLdu3fPFmNmZqZW2Te7eL8Z+4eWe0WtVjNv3jxsbGwAePHiBTKZDLlcztq1a4mJieH48eNMnToVFxcXAgICcl2/dylZsiS2trZYWFgA4OXlRc+ePYGsoQVhYWHI5XJKlixJy5YtOXnyJC1atNDax5o1a3Bzc6N06dKA9nsEsHv3bjw9Pd863ODBgwccP36c1atXM2LECHr37o2VlRVDhw6VhkoIgiAIgiAIwn+RaEEXPkiNGjW4desWV65cAbJmBH+VXL6Ljo4OSqUSgEaNGhEWFoZCoUCpVLJt2zatsqGhoWg0GpKSktizZw8uLi55Fr9KpeLXX3+lePHi1K5dm6JFi5KZmcn169cB2LVrV54d610aNGjAypUr0Wg0KBQKBg4cyNq1a7ly5Qpt2rTBxsaG7777jl69ehETE5Nnx23RogWRkZHSzO3h4eHY29sDULVqVfbs2QNAWloaUVFRODo6ZttHdHS0NMP78+fP2bJlC+7u7tL6v//+m7p16741hp9//pnhw4cjl8tRKBTo6Oggk8lIT0/Ps3oKgiAIgiAIwudItKALWtLS0rI9am3Dhg3S32ZmZsyePZuffvoJuVyOnZ0durq67+0uXrduXYYPH05QUBBjxozh5s2beHp6YmRkhJWVldb2pqameHl5kZ6eTvfu3albty737t376DqtXLmSsLAwZDIZKpUKe3t7fvvtN+lYI0aMoF+/fpibm9OyZcuPPs6HGDNmDFOmTKFt27ZkZmZSv359+vbti56eHq1atcLb2xsjIyMMDQ2lLuh5wdXVlfj4eHx9fVGr1ZQuXZopU6YAMH36dKnru1wup1WrVnh4eEjx2tnZ0bVrV8aPH8/48eNp3bo1SqUSHx8fnJ2dpWPcvn1bmoTuTceOHcPY2Jjq1asD0Lt3b0aNGoVGo2H06NF5Vk9BEARBEARB+BzJNDn1QRY+O5cvX6ZKlSrvLRcdHc39+/dp167dRx0nJSWFhQsXMnjwYIoUKcLFixf57rvviIqKem8r+itHjhzh6dOnUvI3efJkDAwMGDFixEfFJPw3qVQqpk6dyrhx4z51KP9Ibj+7gpBXxDUnFDRxzQkFTVxzQkHLzTWX2+tStKALH8TExAQ9PT1pIjddXV3mzp2b6+QcoGLFiixfvpxly5ahVqupXLkyEydOzL+gBUEQPgNPUzLYHH2P9EwV3jWs+Mrc6FOHJAiC8FlQqTXoyHP/W1TIH+mZKk7fTuQrcyPxHfYPiARd+GD+/v74+/t/9PaWlpb8/vvveRiRIHx+bj5J5c8L8ahTkrGpqEZft2CnBHmcnM6F+0k4WJnxhYlBgR5byC45PZN2C45y/3nWEx+WH7nJbj+Xz/YHzoX7SaRkKKldzlz8aM5H607eZtPfdzEz0mdI04rUsC6eq+2UKjUymUy8N8JnL02hJGDLefZciMfS1ICJ7arRvNqXnzqsPHUk9gkXHyTh/PUX2JUpBsDW6Hts+PsORQ31GOxWkepfmX3iKOH642S6Lj1JQnIGMhkMa1aJH1wrSusPX0tg/+VHVPjCmC7fWGOop/OOvf23iQRdkFy8eJEePXqwcuVKaeKwZ8+e0alTJ8aOHUvjxo3fuu3IkSOpWLEiffr0ydcYDx48yLlz53J8frarqyt6enoYGhqi0WjQaDS4u7vTr1+/986qntfmzZtH2bJl8fT0LNDj5sbevXtZsmQJCoWC0qVLM336dIoXL05qaiqjR48mLi4OtVqNt7d3ju/nw4cPGTNmDE+ePEGtVtOnTx/at28PZM3QPmnSJB49eoRKpSIgIAAXFxcUCgWDBg3i1q1b1KtXj8DAQADu3LnD+PHjWblyZUGegk/u9J1EOi0+hlKd9TrqwUk29K9XYMffef4B/hvPkqnSoK8r59duNWhW1bLAji9kF37xkZScAySnK9kSfQ//ZpU+YVQfTqPRMHDtaf68GA9A5S9N2di/HsWM9FAo1WQoVZgaftwTOARte2IeMmbbBel19O1EjvzUBDMj/bduo9Fo+PnPK6w6dgtduZyBjW0Y1OTrgghXEPLFwsg4dp5/CMCDpHSGbDjLidFu79xGo9Fw5PoTbj1No4mtBVbF8+9GaGqGkml7LhMV+4QqXxZlbJsqH3S8GXuv8GtkHAAyGczu5EixInoM23xOKnPy5jOiAppQ3Pjtn/23yVCqSE5X5smN+rn7Y0lIzgBAo4F5EbF0/caaEiYGhJy+x9BN/4s5KvYJy3vV/sfH/LcSs7gLkmrVqjFixAiGDBlCUlISSqWSH3/8EW9v73cm5wUpJibmnc87nzlzJtu3bycsLIyNGzcSExPDtGnTCjDCLEOGDCmUyXlMTAxBQUH88ssv7Ny5k3LlyjFnzhwAVqxYgaGhITt37mTTpk2sXr2a8+fPZ9vHpEmTaNiwIWFhYaxcuZKgoCDi47N+jA8YMIBGjRoRGhrK9OnTGTp0KAqFgqioKEqVKsW+ffu4f/8+165dA2DatGmMHDmy4E5AITFx+wUpOQc4ceMZF+6//brOa1N2XSZTlTX9iEKpZtruywV2bCFnOfWgeFuvikyVOsflhcGxuKdScg5wJT6Z9X/dYc2J29ScvA+HSeEMXBtNeqbqE0b577Dv8iOt1ykZSo5ef/LObfZefMSSQzdIz1STkqFkxt6rnLr1LD/D/NdQqTWcvPGUq/HJnzoU4TXn7j3Xev0yU8W1R+9+j0aFxOC7/C/GhV7AbdYh/rqZf5+BKbsvs/bEHW4/TePPi/EMXHs619umZ6pYFnVTeq3RwIID1wm/mP2zf+T6kxwfbfwuW6PvUXvyfmpN3k+X346TmKqQ1iW9zCTi8iMOX0tgT8xDnr227m0ev8jQep2p0kjbrT1xW2tdxJXHzNl/Df+NZ2m34Ajz9seiLMTfbQVNtKALWrp06UJ0dDSjR4/G2tqaokWLMmDAAGn9b7/9xpYtWzA2NqZWrVpERERw4MABIGsCur1795KSkoKzszM//fQTurq6nDp1iuDgYF6+fImenh4//vgjDRs2BODXX39l165d6OjoUL58ecaNG4eFhQXh4eEsWrQoqwuejg4BAQHo6+uzYcMGVCoVpqam7+1mb2RkxPjx42natCn+/v6Eh4dLrccAISEh0uuRI0diaGjItWvXePr0Ka6urpiZmREZGUlCQgKTJ0+mXr16H1TuVY+CuLg4pkyZwvPnz1GpVPj6+tKhQwdSU1MZNWoUt2/fRi6XU61aNQIDA9/6/PC8EBYWhre3tzTL+uDBg3n+POvLTaVSkZqailKpJCMjA7Vane357gALFy6UvgQePHiArq4uBgYGXL58maSkJLp16wZkPbZt/fr1yGQy9PX1SUtLQ6FQkJ6ejp6eHpGRkZQqVYrKlSvnW30Lq9jHKdmWHbz6WOq6lp80Gg1PU7S/aBNSMt5SWigozapaUrVUUS49fAFAqWKGdKr1lVaZpJeZDNt0jogrj7AqXoTJnvY0qmTxKcJ9q1etJ6+7kZDC1tP3UP//b8c9F+Kxt7rJ941Fy+0/YWNhkm3ZxLCLmBnp4/z1Fzluc/6NZAbg3L0kapUzz/P4/k2epGTQeclx4hJSAfCuYcWsTtkfQyoUvG/KmRMV+78bU6YGulQtVZQ7Nx7lWP5h0ks2nrorvc5Qqll8KI5vyufPZ+DQ1QSt1zH3k3iWqsA8l63dKrV20n0v8SUdamZvgV9z4jZDNpyh3BfGTGtvT50KJd6538RUBaO2xaD4/9aCEzee8cuBWCa0rUb07Wf0WvE3yRlKqbyOXMbsTo54VC/z1n16OpXhr9du+NmVKUpFS1MAjA2yp5zz9sdKf5+/l4Rao/nseo3lF9GCLmQzadIkYmNj2bt3L9OnT5cmgIuKiiIkJIQtW7YQEhJCamqq1nbx8fGsXLmS0NBQrly5wqZNm0hMTMTPz48xY8awY8cOpk+fzogRI7h79y5bt24lKiqKLVu2sGPHDipWrCi1pgYHBzNhwgRCQkIYMmQIJ0+exNHRkS5duuDu7p7rMfBffvklJiYm3Lhx471lL126xKpVq1i7di0rVqzAyMiIDRs20KNHD5YuXfrB5QCUSiV+fn4MGzaMkJAQaZuzZ8+yb98+UlNT2b59O1u2bAHg7t275Kdbt26hUqkYOHAg7dq1Y9KkSRgbGwPQt29f7t+/j4uLC40bN6Z169Y5Js9yuRwdHR18fX3p0qULHTp0oHjx4ty8eZMyZcowbdo0OnbsSJcuXUhISEBPTw9nZ2cMDAzw9PSkTp06lClThkWLFuU4VOG/wCSHLr5lvzAukGPLZDK8amh/wXrXyPmxeELBMdTTIeT7+szv6sSMDg7s9W+Ihal2l8PZ4VfZf/kRGg3cffYSvz/OFLqW6Ca2JSlu9L/rW0cuo5KlKW/8xuTSgxcFHFnO7j9/yYIDsSyLuqHVepSf9l96xMC10YzZFsPdZ2kfvZ9e9cvhUlE7EU9IUeC/8exbW6LqvvGjXSaDOvmUmPybrDx6S0rOAbaevsfZu9lvdvwbRN9+hsevR6kZtI9xoRfIUBauf2Pe1L9RBbrXtaZYET2qlCrKEt+aOSaDr2QqNbzZ0JyfdbT90lTrdUlTA4z0czf22lBPh3JfaCfjGUo11UoXo75N1mdZRy6jSilT/rr5DLUGbiSkMmj9GSnxzsn1xyls+PtOtjJHrz/l+uNkZoVf00rOIetGQcCW8+9s5e5Wx5rZnRxpVtWSfi7lWdn7G2ndD02+Rk/n3XNehF/K+abKf5FoQReyuXnzJqmpqWRkZHDx4kW++SbrA3bo0CFatmxJ0aJFAfDx8eHEiRPSdh4eHhgZZf1D0q5dOw4dOkSZMmWwtrbG0THrTnPFihWpUaMGf/31F4cPH8bLy0vapkePHixevBiFQkHr1q354YcfaNSoEc7OzvTr1++j6yOTyd77nHaAJk2aoKenh4WFBUZGRri4uABgbW0ttTJ/SDnISojv3Lmj9Yzv9PR0Ll26hIuLC3PmzMHX15f69evTs2dPypYt+9H1zA2lUklkZCQrV66kRIkSzJgxg7Fjx7Jw4UICAwNxdnZm6NChPHnyhN69e+Pk5ESLFi1y3NeaNWt49uwZvXv3ZuvWrejp6XH69Gm+/fZbRo0axfnz5+nXrx9hYWFYWlpKz1uHrJ4THTp0IDExkdGjR6NUKhkyZAhVq1bN1/oXFkObVmTUa2NHi+jp0NahdL4dLzk9k0yVRrpjH+hhh42FCWfuJlK7nDm+dfP3uhNyx1BPh7aOb78Ozt3THgaR9DKTm09SqVKqaH6HlmvFjPTYOrA+y4/cJDVDSddvrKlkacrsfdd4+drNhIYVP33L/91nabT+JYoX6Vk/RFcfv83WgfV4qVBjXSJ/xqRGXn1M39WnpNcRlx9zcETjj5os6XFyBmXMimCoKyf9tR/aj5MzeJKi4Mtihtm2aVjJgrGtq7DiyE30dOUMavJ1gfTc+dw9Tk7PvuxF9mX5IeZeEmNCY4h9lIJr5ZJM9bKnWJH8mcchPVNFv9XRUrfkNSduY26sX6hbNQ10dZjsac9kT/tclbcuYYRr5ZIcuPIYyLpJ1bNeuXyLb3ybqjx4/pIr8cnoymU8Ts6gwfQDTPNyyNXcLzWti3P9sXaDWLEieqzvV5c7T9MwMtCh+7KTWuufpGRwLzGNCjn0sllwIJaZ4VnDDGXA6/cqrj1Kpunsw+i+ZfLIDKWaiw9e4PiOCem8aljhlcNN/zoVSrC4e036rDqVw1ZZKhRQQ8XnQCTogpZnz54xePBgRo0aRUZGBkOHDmXbtm1YWFigq6urNb5FR0f7B8XrrzUaDbq6uqhUqmyPYNNoNCiVStRqtdY6tVqNUpn1Q8nf3x9vb2+OHj1KSEgIK1askFqZP8T9+/dJS0vD2tqaCxcuaMWfmZmpVfbN7txvm1gut+UAqTv+9u3bpWVPnjzB1NQUAwMD9u3bx8mTJzlx4gS9e/cmMDAQV1fXXNfvQ5UsWRJbW1ssLLJ+HHt5edGzZ08A9u3bR1hYGHK5nJIlS9KyZUtOnjyZLUH/888/adCgASYmJpibm9O0aVMuXbpEs2bNKFq0KE2bNgXAwcEBKysrrly5gqXl/76EHjx4wPHjx1m9ejUjRoygd+/eWFlZMXToUNavX59vdS9MQs7c13r9MlPFk+R0vjDN/oP6n5qz7xqLDsWRqVLjbleK2Z0dMdDVoV/DCnl+LCF/1SlvrtVq94WJQY7dnD+1ChYmTGmv/WN5ec9azAi/ytMUBR1qWtGx1qfvtbE5+p6UnAPceZZGg+mRZCjVVClVlBW9alGq2Ptv7n6I7W989uNfpHPixlMa25bM1fYPnr9k4993UarU/PH33RzHhVYsaZJjcv5KX5cK9HURn/8P4Vm9DJuj70ktrxamBjSomPMwgrykUmsYsDZamkByV8xDihbRZZqXQ74c72p8crZr6viNp3z8c3sKp0XdaxBy+j63nqbSvOqX1Cybu6cffIxyXxjz548NGbg2mj0XsubneJKiYPjmc5wc7fbem3N9XCqwOyZeatFuVMlCSpBf3Uj8prw5V16bG+HLooZY5/AEkKSXmcx9rVu5BjArosvLTDUZr93kU77Z5en/yWVQ2uzj/010q2JJ3wbl+f3YLVRqDZW/NOXO01TSMtVU+MKYgJa22ba5cD+JOfuu8Tg5g/ZOZfi2QfmPPv7nRHRxFyQqlQp/f3+aNGlCmzZt8Pb2xsXFBX9/f1QqFY0aNSI8PJzk5Kx/BN5MmHft2oVCoSAjI4Nt27bRsGFDqlevzo0bN6TJxmJjY/n777/55ptvcHFxYevWraSlZXXxW7NmDbVr10Yul+Pq6srLly/p2rUrEyZM4OrVqygUCnR0dKQk/n1evHhBUFAQPj4+GBgYYG5uTmxsLBkZGWRmZrJ37948PHs5K1++PIaGhlKC/vDhQ9q0acOFCxdYv349o0aNokGDBowYMYIGDRpw6dKlfI2nRYsWREZGkpiYCEB4eLg0Y3/VqlXZs2cPAGlpaURFRUk9H173xx9/sHbtWgCSk5OJiIigbt261KhRA319fSIjIwGIi4vj7t272brJ//zzzwwfPhy5XC69pzKZjPT0gmmNKAzO5dA1ctGhuDw/Tsy9JOZFxKJQqtFosn7cbTp1L8+PIxSMH5tWonOtryhWRA9Hq2L81qNmgT+e72PV//oLtn3vzOGAJvi5Vcx24/ZT0MuhlejVj9TLD18w6/9bmfKSZdHsiXNOy3KSkJxB2/lHmBcRy68H47IlUkX05DSqZMFi35p5EqvwP/W//oKVvb+htUMpute1ZsuAehjp538b14PnL7We7gDk64RmNiVNMH6j+7XDv7CHhYGuDl2/sWZUqyrIZTBmWwzT/7zCo3zsFXHziXYreNLLTOKT3n+8SpamRAxvxGRPO5b41mR5z1rZyoxoYYtH9dIY6etgX6YYi31roquT/bshPulltuTbQE8nx2T+TToyGaPdq2QbevWhxrapysnRbkQFNOHPHxvy99hm7B/aiIhhjShbQrsF/aVChe/yk0RceUzM/SQCd15i06n8HQpaWHzSFvTY2FhmzJjBy5cvSUtLo1GjRgwePPitX9wPHjzgypUruLq64uvry8SJE7GxscnTmP7++29MTU2zJRV2dnY4OTlJr21sbJg4cWKu9/v8+XOioqJo27ZtXoWa515N5PbTTz9Jy8aPH0+nTp2YPXs2I0aMoFOnTnTu3BlDQ0MqVqyo1XXcysqKbt26kZqaSrNmzWjfvj0ymYx58+YRFBREeno6MpmMadOmUb58ecqWLcvDhw/p2LEjarWasmXLMnPmTHR1dRk9ejTDhw9HV1cXmUzG1KlT0dfXp27dugwfPpygoCDGjRuXrQ7Dhw/H0NAQHR0dVCoVzZs3lya5c3Z2pnbt2rRq1QoLCwvq1KnD1atX8/Wc6uvrs3DhQqZMmcKyZcukrtw1a9akSpUq/PXXX7i7u1OkSBFKlSqFr69vvsbj6upKfHw8vr6+qNVqSpcuLXU9nz59OoGBgYSGhiKXy2nVqhUeHh4AjBkzBjs7O7p27crPP//M+PHjpWu5U6dONGvWDIDly5czefJkZs2aBcDUqVO1Ws+PHTuGsbEx1atXB6B3796MGjUKjUajNQzg385YXwfFS+0bTbXzYZKmqznMZHtNzED82Sqir8P0Dg5M75A/rWf/NZ1qf8WaE7d5nMPEdpA1TjOv9WlQnn2XHnHj/3+s+9SxzvUQhZ3nH/D0HePkezuXJ6Dlf2/SzYLSqJJFnkzKmKFUcftpGuVKGL/3BltpsyKUKmbIw9cSOadcPuv+Y5gY6DK3ixPjt18g/kU6bpUt8Wta8f0bFmIZShXrTtzh8sMXuFSyoN1rw4jO3Emk4+LjUtIadvYBEcMa5cvzuRvZWmi1cpcrYZSrxBigpKkh3d8xFO3Ri3S61LZmZkdH9HJIzF95c4JYgCJ6unSu/RWTd+X8NBfLogYs61mLMmZGuZ7Y7n1ef6SbsYEuX5fMuSdY9O1EEtO0e7vuv/Qo2wSq/0YyzYfOyZ9HXrx4gY+PD/Pnz6dcuXKoVCqGDBmCs7MzXbt2zXGbkJAQbty4wfDhw/MtQR85ciTu7u7SLOOvODs7c/To0Y/e78mTJ9mwYYP0SKu8dvnyZapUqfLectHR0bYI3RoAACAASURBVNy/f5927dp98DFiYmI4c+YMPXr0AOD333/n3LlzzJ0794P3JQj/lEqlYurUqTneqCnshvxxhu3nHmgtOz2uWZ59+b3yMOkljYIPonhtUpcVvWrhWlk88/y/LLffF59CeqaKXyOvc/LmM5y+MmOwW0VM3jHh0z/1PO3/2LvvsCbPrw/g3yw2Ago4QFAQt6CoLe69V+uedf2q9W3VarWuaq12OGtrW2vd1Vq1Q6271Trq3ltERRRREJAlICMk7x83Schgygj6/VyXl+TJ8yRPBiHnvs99Thr234iApUyKZf/ewaMY3Wzl1E41iqRHuDJDhUuhcShra5HtF1NTtp4LxfTt101e5+Nqhy1jAgqll3FRMOf33Mt6mpCCVf/dR0RCCnr5VULHOhWy3fdUcDQ++PUyYpLS4GJviZVD/dHQM+fB2YsPYzBj+3XcjUxEmxquWNLPr9D/VhhSq9VIVaqKJFAtLpr33Pgtl7E7y9/bmV1rYkxLETvM2nEdm8+G6h23aljDHF/DgkpJz8CiA6LQp7eLLWZ1q52v3//szN55A5syW5h5lrPBtjFNsl3icuFBDPquPK23zc/dAX990Bxrj9/HfIMgXSaV4LcxTdCwStENCuUkLDYZLRcd0SsyOq61N6aZ6UBkXj7n8vpZWGIz6P/++y/efPNNVKlSBYBYv7xw4UIoFApkZGRgzpw5iIiIQGxsLFq2bInx48dj1apVSElJ0c5kL1++HLGxsbCwsMCiRYtQtmxZLFiwABcvXgQAdO/eHcOHD0dYWBhmzZoFpVIJiUSCTz75BDVr1sT06dMRGhqK1NRUjB49Gh4eHjh+/Dhu3ryJatWqoVKl3Is2LV26FDdu3EBSUhK8vb3x1Vdf4dmzZ5g+fTqeP38OtVqNhQsXYuXKlbh9+za2bduGZs2amTyfNm3awMvLC15eXmjcuDFWr14NuVwONzc3LFq0qEjbb+VF1apVsXr1avz222+QSCSoWLEi5s+fX6LnRFQahRmkLAJi7V8T75zbouRXRQdrrB3RCMv/vYvnKUoMa+LJ4JzM2qd/3dS2QDoXEoOw2Bf4YYh/kd2fo40FBr3hAQBo4OmExX/fRkh0MjrVKY/3WhXuBICGXCYtUEun7n6VsOZEiHZm38vFFssHNkCqUoUGlR0hzaawU0FoBkrOhcTA39MJ49tWK5aU7tImQ6XGwFVntOnLe6+FY+VQf3SuW9Hk/p/suKFdmhD1PBVz/rqJvRNa5HgfDT3L4p9JraBWq4ttaYhEIinVwblGQko69l7THwzfcu6RNkA3VWyvqArwWSlkmNOjNub0eLliuOdCYnDgRgQql7WGv4eTNjgHgIfPkrH6+H3M7m76PgxbtgEiS+NcSAyeJaVBIZMgPUOtt/+Je9EvHaD/deUx9l+PgLuTNca28tZLk09KVeL7I/dwLSwOAVXLYWwrb21mibuTDWZ0qYUl/wQhValC4ypOGPua1M8psU/byMhIVK6sn6KgafcUFhaG+vXro1+/fkhNTUXLli3x4YcfYsyYMbh//z7atWuHDRs2oGPHjujWrRs2b96Mn376CQEBAQgLC8Nvv/0GpVKJwYMHIyAgAD/88AOGDRuG9u3bIzAwEDNnzsTGjRtx9uxZ/PnnnwCAkydPom7dumjRogW6du1qFJzHx8frpR9PmzYNVapUQZkyZbB+/XqoVCp069YNT58+xerVq9G2bVsMGjQIp0+fxrVr1/Dee+9h69atGDBgACZMmGB0Ptu3b0d4eDi2b98OJycnTJgwASNGjEC3bt2wc+dOJCYmaqunlxQ7OzssX768RM+B6FUQ/yLdaJuVomgG4Fr4uKCFGVTMJsqLfdfD9S4fuBkBlUpdqMFndqo622LFEPNdv21nKcee8c1x8NZTqAF0qFUe1nls15Rfs3fewO8XRb2KsyExeBL3At8ObJDLUa+fy6GxRmuL/7z02GSArlar8eCZ/r6Gx+akMIPzxFQltp1/hCdxL9C1XsUiLZJWkixkUlgrZEhK03WQKJMlAB/etAr+uvJEu86/XU3XAvdDT89Q4X5UEjzL2RTZ4MY/NyMw9peL2mKFplLkc3pPNapSFt4uttqWgRIJ4FrGEv1/Op3tMYHhL9cS84+LYZjy+1Xt5RP3orF/Ygvt+3nK71e1xfNO3nuG6MRUfNarrnb/d1t6YcAblRGfnI7KeVwS8CoosQC9UqVKRgWxHj16hIiICNSqVQvXr1/HmTNnYGdnh7Q002uuGjUShRL8/f1x7NgxuLi4oFGjRpBIJFAoFPDz80NwcDCCg4PRuHFjAECtWrUQEREBOzs7zJ49G7Nnz0ZiYmKuKd8ODg7YtGmT3rb09HTExMRg8uTJsLGxQXJyMtLT0xESEoK+ffsCAJo0aQJApLhrmDofAHBycoKTk/iQnDFjBn766Sds2bIFXl5e2srYRFT6mWrPExyZWKRrC6nwpSozcDQoChZyKVr6uECWQxB59+lzLD98D88SU9HH3x19GpZ8FXNz5OZkrbdOs5KjVbEE56VFbq34Csteg4GSvdfCGaCbYGpJgWs2RbQkEgk61C6Pv2/qej13KoJU6rx4Z+1ZXAoVxUrXnwzB+pFvFMr6enNjpZBhUofq2vXVFnIpPsrSMq58GSv8+1Er/HcnCg7WCrxRtWyBBkKuPorDmE0X8DQhFQ7WCnwzsD7a5LEzQ35sOvNQr4d7aEyy0T5ujtkXnZRJJdg2tgl+PvUAkQmp6NWgEqb/aXrZjEaL6vnvVpCqzMCPR4NxLiQGDw0GpW5HPEdg+HPUrlQGaUoV/r4ZoXf9rqtP9AJ0AChjpUAZq6LJbDBXJZYz3aZNGxw/fhyhoWLtR3p6OhYsWIA7d+5g+/btsLe3x9KlSzFq1CikpKRArVZDKpVCpdKtpbx+XbypLly4AB8fH3h7e2vT29PT03H58mV4enrC29sbFy6IvnuBgYFwdnZGZGQkbt68iR9++AGrVq3C4sWLtSnneV2W/99//yE8PBxff/01Jk+erD1Pb29v7bmdP38eixcv1jt3U+cDQC+Ffdu2bRg/fry2WvbBgwcL9kTnQ1hYmF4hvKy+/fZb7Ny5M8fjt2/fjrFjx5q87ujRo/j2229zPYdhw4bhwIEDuZ9sHn333XcICAhAr1699P5pXp/8+v3337F58+ZCO7+S8Pfff6N3797o3r07xowZo63onpSUhIkTJ6J79+7o2rUr1q5da/L48PBwjBo1Cj179kT37t2xY8cO7XXHjx9H79690atXL7z99ts4ceIEACAtLQ3vvvsuOnTogDlz5mj3Dw0NxYgRI4ruwZqptCztTDQSUoxn1Sl7j+NeYPHft/HVvkDcjyr8Yl65iUtOQ5dvjmPsposYuf48+q48ZfJ1BUQl2kGrz2D31Sc4FfwMH/1+1WimuKhdC4vD53tu4adjwUhKM32eRUGZoTKZVpmduT3raFNM7S3lmGfwRY2Kh7uTdY6XXwVHgiKx5O8gHL79NPeds1HF2RajmunaPlVysMpxacSivn54p4kn6rk5YFSzqvj8reJ/f994HK8NzgFApQZ+yZIm/ar5XwsvHJrcEt8NaoAT09qgpcFAhJVCho51KuBNr3IFzlL4bPdNPE0QxSbjX6Rj1vbreY4l8sPGIGPG1Nn28HMDIHqa770WbtTtwdnOEh91rIGFfX3R1NsZhg/ZUi6Fv4cjPMvZYFL76hicuQQoP+btvoVvDt3FqeBneBynPyEhk0rgbCdqKChkErgatJdNSs1AdKLpwp2vkxKbQbezs8OCBQvwySefQK1WIykpCW3atMHgwYNx7949TJ48GRcvXoS1tTU8PT0RGRmJ6tWr48cff0SdOnUAAIcOHcLPP/8MW1tbLFy4EA4ODjh37hwGDBiA9PR0dO7cGXXq1MHHH3+M2bNnY926dVAqlfjiiy/g4uKCqKgovPXWW7CxscGoUaMgl8vh5+eHJUuWwN3dPdcCdL6+vlixYgX69+8PCwsLVK5cGZGRkXjvvfcwc+ZM7Nq1CwC0Fcjv3LmDDRs2mDwfU7c9cuRIODo6wtbWFq1bty701yA/Jk6c+FLHX79+HfHx8YV0NvnTtWtXvaDwZVy8eBE+PqW3oun169cxf/58bN26Fe7u7vjyyy+xbNkyzJs3D+vWrYOVlRX27NmDxMREdOvWDY0bN4avr37F6M8++wwtW7bEiBEjEB0djY4dO6JJkyawtbXFlClT8Msvv8DHxwe3b9/G0KFDcfToUZw9exYVK1bE6tWrMXr0aNy5cwfVq1fHV199henTp5fQs1Fy0lXGAZKcs4R5FpOUhl7fn0B0ZkXazWdDsX9ii2JNf/vjYpi2EjcAXA6Nw8FbT9HN1zi19WzIM+25auy7Hg47SzlWH78PQHyJLKoZrLP3n2HwmrPaQNm7rAUO1qtdpDPTarUaX+2/jY2nH0Ahk+KDNtUwNg9rugO8yuHMjHa48/Q5qrnawbYIC8TlJCQ6CTsuP4a9pRz9GrnD0aZoi3KZm7k96mDsLxfxPEUJe0s55vasU9KnVKi+P3wXS7K00ZvUvjomFrBa+ZwetTH4zcqIiE9F46pOsJRnn97sYK0o8UEnU8siDAO/V001V3tUc7Uvstt/+Ex/Jjs8IaVARfYyVGos/ScI2y89hou9JWZ0qYmm1XQz2ONaV8Pxu9FIzkzZ79fIHWlKFXZdfQKZVIJRzavijapl8d2/d7H0oHh/21jIsGHkG9mm7o9r5a1XfHJsK29MzpJlkB/3Ip/j4K1I7Lj82OT1EgnwfptqcM1sLSmRSDA0wEPvdzEtQ4WNpx8W+BxeFSVa8aNu3brYuHGj0XYfHx/s3r3baHv58uW1vau7detm8jaztgjTcHd3x/r16422z5s3z2jbwIEDMXDgQKPtpiq4u7i4aNewG1q5cqXRNk2PaQAmzyfrfbRt2xZt27Y1edslYfr06fDx8cHo0aNx7NgxLFmyBFKpFLVq1cKpU6fw66+/AgCioqIwZswYhIeHQyaTYenSpUhMTMTWrVuRkZEBe3t7TJgwAYsWLcLhw4dhb28PX19fBAcHa5cQHDx4UFsQsEePHhg3bhzCwsIwfPhwNGvWDDdu3EBGRgYmTJiAbdu24f79+6hbty6+/vrrfBXSi46Oxpw5c/Ds2TNERUXBzc0N33zzDcqVK4eQkBDMmTMHMTExkEqlGDduHBQKBQ4fPoyTJ0/CysoK/fv3x4IFC3D69GnIZDL4+vpixowZsLOzQ9u2beHr64ugoCBMnjwZUVFR2Lp1KxQKBSwtLTFv3jxUq1b41YFzs2vXLvTp0wfu7iK9dvz48YiLEyPpGRkZSEpKglKpRGpqKlQqFSwsjL+UrlixQjsy/OTJE8jlclhaWiI9PR2ffvqpdgCjWrVqUKvV2kKOycnJSEtLQ0pKChQKBY4cOYKKFSsatTR8HZgaWL8ZzvZneXXgRoRewJuYqsSOy48xoV3xDZ4lpCiNtiWmms6CMDVwYGspx6gN57XtfU4HP8PeCS1Qo0Lhf4n89Vyo3ix2cEwaLobGFklrP439NyKw6j8x+JCSrsJX+2/jjapl87SMw9pCBr/KjkV2brm5+/Q5ev1wUvsl+NdzYgDoVSialVdNqznj7Mx2CIp4jurl7YttoORRTDKuPIpDQ08nVHIsuln7tSdC9C6vOXG/wAE6UPQBYGHydrHDW/UrYecVUTzNzlKOMfkovPUi8/eiqOoflEYd61TAlnO6avAtfVwK9Hnxy5mHWHE0GAAQkZCC/228gNMz2mmziupXdsTRqa1xNCgKHmVtEOAlCsvO6VEHcpkEZawUeJ6Sju+O3NPeZnJaBr45dAe/vhtg8j4HvuGBGhXsceZ+DOq5OaC5T/5T2gHg5L1ojFh/Tq/InEYlBytM6lAd2y+FYffVJ0hOVWJq5xqwlMtMVrJ/xhn0kg3QqfSJjY3Fxx9/jJ9//hk1a9bEjh079FKcHz16hGXLlsHT0xOff/451q5diy+//BIDBw5EbGwsJk2ahK1bt+LmzZvYs2cPJBIJxo0bp3cfSUlJ+O2335CSkoJ+/fqhdu3a8Pb2RlhYGFq1aoV58+bh008/xRdffIFdu3ZBoVCgXbt2uHLlCvz9jav97tu3T7v0AQA6dOiADz74AHv37kX9+vUxZswYqNVqjBkzBn/99RdGjRqFyZMno2/fvhgyZAjCw8MxbNgw7Ny5E23btoWPjw+GDBmC5cuXIzIyEn/99RdkMhlmzZqFRYsWaQd+fHx88M033yAjIwN+fn44fPgwXF1dsXPnTly8eLFEAvQHDx6gRo0aGDduHB4/fozq1atr+4//73//w7Bhw9CiRQskJiZiyJAhJoNnzSDIsGHDcPHiRYwYMUJbO6Fr167a/ZYvX44qVaqgcuXKcHNzw4EDB/DWW2+hc+fOcHNzw7Rp07B69epieNTmx1IuxYt0/Vn0DrVYXT2vbC2Nv/gUZSsuU95u4IY1x+9rgzhnOwt0rG16Pam3ix3eb+ONlcfuI0OlRl23MnCxt9AG5wCgVKlxKPBpkQTo1ia+KJraVpiuPooz2nYtLL5U1FnYdv6R9nUFxGz60aAodK5bMuuFi0uaUoWvD97BwVsR8HKxw4wuNYv19fr9wiNM+/MaVGqRUfTtwAYmM1KyCop4jr3Xw1G+jCV6N3DPc9AoMxjMz6l39Kto2YD66N+oMh7HvUCbmq55as+nVqvxxd5AbDwt0uHfaeKJT7KpFv66+bRHbThYK3D6/jPUcyuDKR1rFOh2TgVH611OTsvAibtRUKnFMpMGHk5wtbcy6gOete3ei7QMo+VWpgrTZtXAw+mlf9fXHL9vMji3sZDhs5518OX+29oCdmtOhEAhl2Ja55po4eMCV3tLRD4XQblMKkFvf12NlpT0DJy8Fw0Xe0v4upfcwG1xY4BO+XLhwgV4e3trA7e3334bn3/+ufZ6X19feHp6AhAF8EytnT927Bh69eoFS0vxB2HAgAF6Bfj69u0LuVwOOzs7dOrUCadOnYK3tzcUCoU2q8DDwwMNGjSAnZ0YeXN1dc02hT67FPfhw4fjwoULWL9+PR48eIC7d+/Cz88PcXFxuH37Nvr16wcAqFixIg4dOmR0/H///YdJkyZBoRAjm8OGDcP777+vvV5TxFAmk6Fz584YOHAgWrdujebNm6NVq1Ymz7WoKZVKHDlyBBs2bEC5cuWwePFifPLJJ1ixYgXmzZuHZs2aYfLkyYiOjsbIkSPRoEEDdOrUyeRtbdq0CTExMRg5ciT+/PNP9OnTR3sfCxYswH///YcNGzYAEEF91qUcP/zwA/r27YvY2FjMnDkTSqUSEydORO3ar8cfe0u5zChAj+KIcZ51qlMBvu4OuBYmfue9nG3Rx794i65VdbbFX+83w9bzj2Ahl2LImx5wyqE38dRONfFOkyqITU5DzQplTK5B9yxXNCn6/2tRFftvRGi/pDX1sEFdN4dCu/2U9AysP/kAN57Eo3k1ZwxsXBkBXuXwU+YMOiBSGwtaHbm4aVr8ZGVZRF0WzMnyf+9i5TExexcclYTgyET8+1GrYmvttfBAkLbfsVKlxuK/b+cYoF94EIPBq88iLUN8lu649Bh/jGuap/sa37YaPt11U3s5u373GSoRlP524REcrBWY3qVmsRTpK2oSiUQvdTovjgRFYk2WzIM1J0LQtFo5tu6EWMc+vcvLZwPWreSgV0QQACZuvQxNvD0swBPzc6lb4FrGCm1ruuLw7UjttoGNK+dwROEwteL+6/5+6FC7PKIT04yqyx8NisK0zjVhaynHn+OaYv3JB4h/kY4BjStruwo8iklG35WntOv7+/i7Y2l/v6J+KGaBATrli0wmMyp8kTWtXC7XvaWyK7iXdR/D4zX3oaFWq7X7KxQKvS8KmsC4oBYvXoxr166hT58+ePPNN6FUKvXuL+t93b9/36j1nkql0ttHpVIhPV03Smljo/uyvWTJEty5cwenTp3CqlWr8Ndff+WpaF5hc3V1RY0aNeDiIta69u7dG8OHDwcglhbs2rULUqkUrq6u6Ny5M86ePWsUoB84cADNmzeHnZ0dypYti/bt2+PWrVvo06cP4uPjMWHCBKjVamzbtk07s57VkydPcPr0aWzcuBFTp07FyJEj4e7ujsmTJ2uXSrzqXOwtEWcwou35GrUPeVlWChn+HNcUx4KioFSp0LqGa4mkH/uUt8+236wp5ctYoXzm2rtOdSqgh18l7L4q0ky7+1ZE5yKq6FzN1R5Hp7TGv7cj4WpviXLKqEK9/Y9+v4q918SAw95r4YiIT8GkDtUxs2tNrD/5ABZyKca39UGtiiXbKjSvhgR44rcLj7TLKBp4OKJFPoOZ0uhIUKTe5fvRSQiJToKXi3EKamFTq9VGhTINPyMNbTz9UBucA8CFh7G48igO9fOwPGJ40yrwq+yICw9En3f/bGYPt5wLxbqTIihNTFVi0rYraFTFCRUdXr2iebm59cS43VZg+HMG6IXofy28cP5hLP67o/uMzjoZ/svZh3ivtTfccln+8cNgf2w8/QB3IxPRtqYrutbLOROlMLzTxBNHg/T/ttx6koDe/u5QyKQoYyXXWxpWo7zuc6VyWRuT/eHXHL+vDc4B4M9LYXi3ZVXUrFA6/pa8jFd/SJgKlb+/Px48eIDbt28DEBXBExISch1hl8lkUCrFL2arVq2wa9cupKWlQalU6qXIA8DOnTuhVqsRHx+P/fv3o0WLFkXyWE6cOIHhw4fjrbfeQrly5XDq1ClkZGTAzs4OderU0VatDw8Px6BBg/D8+XO9x9GiRQts2bIF6enpUKlU2Lx5M5o1a2Z0PzExMWjVqhUcHR0xYsQIfPjhhwWuIv+yOnXqhCNHjmgrt//zzz+oV68eAKB27draOgnJyck4fvw4/PyMRyq3bNmi7S7w/Plz/PvvvwgICEBGRgbGjBkDd3d3rFu3zmRwDgALFizAlClTIJVKkZaWBplMBolEgpQU49Zjr6rPetZB1vpc1cvb5Xs243WnkEnRvnZ5dK5bsVSuDZZJJfhuUAOcnN4WJ6a1wfeD/SEvwjRbJ1sL9G3ojpbVXSAtxBnRpFQl9htkA/yR2T97TEtvnJ7RDsemtkHfUtRWzs3RGocmt8LCPvWwYog/to1pUqSvjbnwMVgLam8lL7ZAVCKRYIBB2u7AxjlXj5bLjN/H+Sm2Wb+yI/7Xwivb4BwALj6M1busVKlxJdR4+cbroJmJv1GmtlHBWVvI0MffLdvr1WrgeR46vlhbyDC2lTeW9PMrluAcgMnPivOZvz9WChkW9/PTpuL7ujtgWh4yDmKSjR+rYVX6VxVn0ElPcnKyUau1rVu3an92dHTE119/jWnTpkEqlaJu3bqQy+Wwts75j3hAQACmTJmC+fPnY9asWQgJCdFW0Hd3d9c73t7eHr1790ZKSgqGDh2KgIAAhIWFFe4DBfD+++9j0aJF+Pbbb6FQKODv769t+7d06VJ89tln2LRpEyQSibbyf8uWLbFgwQIAwLhx47Bw4UK89dZbUCqV8PX1xezZs43up2zZshg3bhxGjBgBKysryGQyvWUBxalt27aIiIjAsGHDoFKpUKlSJW3q+cKFCzFv3jzs3LkTUqkUXbp0Qa9evQAAs2bNQt26dTFo0CAsWLAAc+bMQY8ePQAA/fv3R4cOHbBnzx5cuXIFycnJ2nR3AFi0aBFq1BDrsU6dOgVbW1vUr18fADBy5EjMmDEDarVauxb+ddC0mjP+/ag1dl99AsmLWIzt3KikT4lKSG4zIebOQi6FvZVCb42jpoVOaeZoY4EBuQSIr5qPO9fEvahE3HicAEcbBb54q16xFgL7tEdt1Khgj8uhcXizatlcB3VGNauKv29EICmzXkCbGi6FunQDAPw9nfQqUsukEviWYAHDktTAwwlL+/nhp//EMoixLb3zlK1A+dO4SlkoZBKT67n9PRzNdva4qrMtnGwUiM0SVPt76N4fnepUQJsarohLTtNWcc9Nv4bu2HPtibawblVnW7xRhMVNzYlEXRSN+qjYBQYGolatWrnud/HiRTx+/Bg9e/Ys0P0kJiZixYoVGD9+PKytrXHz5k2MHTsWx48fz/M6tRMnTuDZs2fa4O/zzz+HpaUlpk6dWqBzotdTRkYGvvzyS5ODIqVJXn93iQpLYb/nfj0bik92XodKLYrPrRneiDNrpVh4/AuUtbXIsV1YfhXV51x4/AscvPUUrvZWaF/LtdAzHTJUaszfcwvbzj+Ck40C07rURK/62c9wkvkozX9bD956iqX/BCHhRTqaVnOGQiaFu5M1hjXxRBmrl1veWZROBUdj9s4bePgsGZ3qVMCCPvVg/5Lne+peNLZffgxXe0uMaFbFqG+6OcnLey6v70vOoL9mZDIZMjIyct8xG3Z2dlAoFNpCbnK5HN98802+isj4+Phg7dq1WLNmDVQqFWrWrIm5c+cW+Jzo9aRUKvXqFRBRyRj8pgdaVnfG7fDnaOjplGOxPDJ/pWl9dUUHa7zTpEqR3b5MKsHcnnVeuT7wZN461C6PDrVL39r+pt4iO7BQb7Oa82u5BJAB+mvGzs4u22rneTVp0iRMmjSpwMeXL1/eZB94ovyIj4/XVvEnopLl7mQDdycWOiQiInpZr37VE9Lj6emJiIgIJCUl5b4zkRm7ffs2fHx8Svo0iIiIiIgKDQP014xCoYCPjw/Onz9f0qdCVGApKSm4evXqa9M3nYiIiIheDwzQX0Pt27fH1atXcfLkyZdaj05UEhISEvDLL7+gWrVq8PB4vaosExEREdGrjWvQX0MODg4YPnw4duzYgZMnT6J69eooV64cFApFvoq9ERUXlUqFlJQUPHjwAJGRkWjcuDHatGnD9ysRERERvVIYoL+mHB0dMXLkSCQkJCAoKAjx8fFITEws6dMqcbGxsXBycirp0yADEokEa8HfGgAAIABJREFUVlZWaNasGby8vCCX86OLiIiIiF49/Jb7mitTpgwaN25c0qdhNkpz30wiIiIiIirduAadiIiIiIiIyAwwQCciIiIiIiIyAwzQiYiIiIiIiMwAA3QiIiIiIiIiM8AAnYiIiIiIiMgMMEAnIiIiIiIiMgMM0ImIiIiIiIjMAAN0IiIiIiIiIjPAAJ2IiIiIiIjIDDBAJyIiIiIiIjIDDNCJiIiIiIiIzAADdCIiIiIiIiIzwACdiIiIiIiIyAwwQCciIiIiIiIyAwzQiYiIiIiIiMwAA3QiIiIiIiIiM8AAnYiIiIiIiMgMMEAnIiIiIiIiMgMM0ImIiIiIiIjMAAN0IiIiIiIiIjPAAJ2IiIiIiIjIDDBAJyIiIiIiIjIDDNCJiIiIiIiIzAADdCIiIiIiIiIzwACdiIiIiIiIyAwwQCciIiIiIiIyAwzQiYiIiIiIiMwAA3QiIiIiIiIiM8AAnYiIiIiIiMgMMEAnIiIiIiIiMgMM0ImIiIiIiIjMAAN0IiIiIiIiIjPAAJ2IiIiIiIjIDDBAJyIiIiIiIjIDDNCJiIiIiIiIzAADdCIiIiIiIiIzwACdiIiIiIiIyAwwQCciIiIiIiIyAwzQiYiIiIiIiMwAA3QiIiIiIiIiM8AAnYiIiIiIiMgMMEAnIiIiIiIiMgMM0ImIiIiIiIjMAAN0IiIiIiIiIjPAAJ2IiIiIiIjIDDBAJyIiIiIiIjIDDNCJiIiIiIiIzAADdCIiIiIiIiIzwACdiIiIiIiIyAwwQCciIiIiIiIyAwzQiYiIiIiIiMwAA3QiIiIiIiIiM8AAnYiIiIiIiMgMMEAnIiIiIiIiMgMM0ImIiIiIiIjMAAN0IiIiIiIiIjPAAJ2IiIiIiIjIDDBAJyIiIiIiIjIDDNCJiIiIiIiIzAADdCIiIiIiIiIzwACdiIiIXl8x94HA3UBSdEmfSdFICAdu7gCeBZf0mRDl3ZMr4n2bHFPSZ5K7pGggJcF4uzIVeHwRSIk3vs7UtqziHwP3DuluNzkGUGXk77xSEoC7h4Cnt3XbngYCuz8EDn4KvMhyDk9vAhHXTZ/HhXXiXNTq/N2/oeQYIEP5crfxmpCX9AkQERFRIVGmAieXA4/OApXfAJpOABRWJX1WhePpLcDaCShTseC3cfcQcO4nQGYhnpvwq8D+jwGoAbk1MGgL4N2m0E65WEXdEQGNnSvg2x+wsAWCDgC/DQMy0gBIgG5LgMb/K97zykgv3vsDgPQUQCIB5JaFc3tqtQherB0BR4/s91OpAGkpn/tKjgGu/y6CwXr9ADuX4j+Hv2cBp78XP1uWAYbvAio1KP7zyI0yFdg+Brj1FyCVAQH/B3ScL657fBHY3B9IjhafLW//CNR5G4i8DfwxEoi8BThVBRoOB0LPAGUqAc0niffXpt5A8L/idmQWYr/oIMC+AtBkAhC0F3h0DlClAzIF4FIb6DAX8G6rO7eQ/4DN/QBlirjsXANoORXY/i6AzED77E/AeyeBv2cCdw+IbbauQAU/IOwsYGkPJEYBqjRxnd8g4O2V2T8fSc8AlRKwL6+//dBc4NR34joLO+CtH4HaPV/iiX/1SdTqlx0OIXMQGBiIWrVqlfRplHp8Hqm48T1HheqvD4DLm3SXGwwFev2gt0upe88lxwC/9AGeXAIkUhFYd/gs/7fz+BKwph2gVonLcmvxpTotUbePW0Pg3cOFc96FJTkGuHMAsC4LVGkBBB8CpHLAp6P4cg4AYReB9V2AjFRx2b0xMPogsLI58PSG7rasHIGpwYAsH/MzKQnAtW3AizigXh+grFfejgu/Bux4D4i8iWRnP9gM3Qw4Vs77/ebVw1NiRvDZPaBGZ8C2PHDlF/EcNR0PtJn5crefHANseksM5kACBIwDOn+lv8+N7SKoTIoCfDqIoDIjDWg8GqjSPH/39yIWuL0PsCoDVO+se42Lw4s4YGULID5UXLawFe87h8pA+7mAx5tFd98ZSuDwfODGH0B8mP51NbsDAzfn6+buXjwGn+h/gMRIwG8gUK1dIZ5spqwDCRpdFgPujYCNPYHU57rtCltgeiiwrqMI3k1xqgLU6gmcWp7/c5FZAGOPA+dWid/5mBAgKVJ/H4kcUBvMYEsVItDPqz7rxOeAoX0fA+fXiM/XOm8DvVeJ927QfmDLQP195dbA1HuApV3e77cUyMvf1rz+/eUMOhER0avi+h/Glw0C9FLnzI8iOAfEl7+T34gv3K75HGQI3K0LzgFA+QKARH+f5GcvdaqFLuY+sKa97rwUNkB6svi5gi8w+h9AYS2+GGuCcwAIOy+yKF7E6d9eWmLmrFsev/5lpAPrOonZPgA4sUwMYLjWzP3Y7e8CUSK11ib6KrB3MjDk97zdb14p04DfhusCkdt7s5x7GnBsoRjUqNqi4PdxdmVmcA4AauDMCqD+YKBCPbEpMRLYMTYzSwFA0D7dsYG7xPNV0S/n+wg+AvwzG0gIE7P/yhdiu0cTYMReMZBUHG7t1AXnAJCWJP7FPwJ+7QdMulV0QdWp5eJ325Tc0sENKdPgeXgckPRYXL7+OzDkD8Cnff5uJzIQiH0oBlkMH/eBGeK9YOjEUvEaZg3OASA9SbxPte8lE2IfiJnmgshIAza9DTx/kv0+hsE5kL/gHAD+HCWOqdUTSIwQA3anvhOZSRo3t4uBqvqDgdDTxrehfCEea4W6+bvv10gpz8MhIiIirTKVcr5cGsU+yNu23Dh5Gm/zCNC/3GBY3m8vMVKkiF7aJIKYonB2lf6ggSY4B4CIa8CtXeJnUwGcVAH4v6O/rV5/EdDnVfARXXAOiCDj4vrcj0tL0gbnWtnNGr6MuIfGs4SGIq695H2E5rwt/JouODekUorZ9ZykxAPbhgJPr4vZc01wDojg5v6R/J9zgUmyvyolvmheQ43gHDJX/Ifn77ZCT8FCE5wDANTA1V/zdxsHPwVWBABbBgDf+oolNhqJUeJ335SUBCAlzvR1L2KBMm653PFLJDbnFJwXpqNfAUtrAMsbAMvqAv98YrxPVJD43/0N07fhUATZNK8QBuhERESvis4LAHnmmnO5lbhc2tXupX9Zk+qdX74D9ddo1usPDN0OdJgvfn5rJdBySt5uK/Yh8MObYv36rg+AtR3FbG5uXsSK9arheQwaswbkpqRlztK9+Z5Y26nh1Rpwbwi0+lg8Lr9BQOeFQI9v83a/GiYD/zzMvlvYAhXr62/zbJa/+84LpyqAXYUcdpAAVVu+3H3UeVv/sk05/dt08xcpu9mxz6VmwpMr+sssDKWn5H6OhaXO2+I5NUWqAFzykDlhikol1vAnRmW/T3mD2VSpXPxeDvkD8O2Xv/uzNbFu3tS27CQ80U8zT34m1mlrKFMAdTYF22p2R7YDHdZlgTfGZHOnEsDKIe/nWJJiQ4HUzOJ18Y9M73NzpwjSa3UHbAyee4X1q1MbpYgwxZ2IiOhVER+qKwqkTDE9+1fa1OoOvL1KrCu2cRZBdEHSbBVWwLAd4kujTKFbS91sQv5v69LPwIss1aWf3gDu/g3U6pH9MU8uAz/3AlIz03WbTtAVlMpOwxHA1S26GVqJVJemb+MM1H5L/FyhLvD+OeD2HlEkrmb3zP0lQP1B4l9BeLUG3BoBjy+Iy9ZOYl11XvRdB+yeCIRfxXPnBrDv9nXBziEnMgUw4BeRPv/sHlCjC1CpIXBhrbiuxUe6VPSCqt4J6LseuPwLYFNW3Kalve56m7JA/41iFjExQjxHmgwPt4aiDkROytcBZJb6SxQ0ynoD5bxFunTW+ywqVmWAsf8BN/4EUhNFobL7R0Xg2PEL4+Jf2UlLBq5sFp8/nk3FbHR0kAjy280Gmk00PsZvkChymBgh1mv3+NZ0YP7wtFhnLZWLegBu/sb7lK+DuKo94BiyW1x2qAw0+SD7841/DISdAyr5i0ybF7H6y2EAkcnw32JRaM2xMuBcHYi+o7teZikGbnosF6/pobkwmg2PDhK/U4Z8OgJdFwPru+Y/nV/DyQuIvV+wY/NNlfsucQ+AVW2AXt8BPZaJpSiaQY1W0wqvgOMrikXiXhGlruiPmeLzSMWN7zkqVEuqA4lPdZdtXYGpd/V24XuuEBz6DDhhEHD232g825/V1iEigNaQyIC2s0Uas7OPCPxsyhof9/SmKNJmUw6o0lKsE5bKRfXnnCqKFxZlqli//yJWPD4717wfe2kTcOcAoiXl4NxznqiCbihDCRxfKgK6jDSR5dDu07wHg+Yo4rqY+XZvJAZJcnNju1jTnBQlipm51BRFv4L2iSUGClug85diwKa4vYgV9y+3yPsxG7oDD45nXpBAL1CVyoFJN0VFcg1lGvCdv/5sbL8NYg3+rV1i0KN2T1GTYWUL3bpphY0YmDJRfDAwMBC1nJTi87BKi+wDwps7gT9Hi+UIgJjJ7/wVcHCOGFTLSmELTAsRj2lhFbHkw1DLqUDbT4AjXwDHFum2y62ACVeBhyfEZ8fTm5nbrUW2iXtDQGYFnFhi+jxLs5EHxOv94ARQ0Tf3mgylFIvEERERmavkGLH+toJv8VepVRrMwpmalSspGUpRvMzWBXCuVtJn83L8h4lZWs1sl3MNwKdTzscYFmxTZwD/zhU/3/0bCLsAjP7b+LjydYAO83SX3XJpN6VW5y0ozCu5pZiFPjxfzCLXeVvMgOZ2H6e+065NdQaAF/eBEXuM9zu2QMxMalzZLJYAjDtRaA+h2OV31r5ub/G8qpS6qu27J+rW/6cniSrZtXqaHsQBRKuu+0dF8FO9c+G9B6yd8rf/05tZgnPAaBZZpRSBdtYA/dFZ41Tpiz+L50DzO3buDTF4k7WoWXqyKAwY8J7pc6lQD0Dma5ESLwaCIq6LWeyA90WxxH8/0wXngMiG+bkH0PwjIOYBkBKruy4jTewrkWVfd+DGnyJAbzQaOPW9bpmKzBL4Y4SuaJrMUgxAhBwFHp0W/yxKUVVz+0qZg4bNgbM/5rxv0D6RLVS2qm5b4B5Rz8LSXrSXe0WD9oIq9DXoZ8+exaRJk4y2T5o0CWlp2a/PatbMeG1SXFwcdu/eneP9mTouP86ePYsmTZpg2LBhGDp0KAYOHIh9+/blfuBLyu35ICKiUujmTuDrWqLl1de1gZDjuR9TmALG6V9+c5zp/Yrb8whgxZvA+s7A9w2BfVNN76fKQ+qkOSjrBYw7LQLnbl8D/zuU+5pKw4JtClv9y4/OvNyShOcRwMZewGdOwI/NxdrmnISeAQ7MBM6szLnIXUa6qA4duBsIvwIc+jT7AllZXd2mf/nBcZFKbCjQRND+9LroF/06kUj0W6pF39O/PiNVFMUz5dJGUW3/6FeipZWpol35kaEErm4VadoPT+XvWHke1hbvmqCfym2qmGVytP4+YeeMK6MbHvs8QmQjGD53APDHaODkt6IY3cE5YsAJEBkCppz4GnjjXf1tDYaI+goKK5HBYoqDu/g/aJ9+DYnUeP2K5hmp4vcpq5xqEZibur3FIFqXBYBnLq0EnavrXw75D9g2BLh3SCxr2NBdDGxrqDLEQONrrNiKxC1btgwWFvlIjwEQFBSEw4eLvh9pQEAANm3ahF9++QVr167FmjVrEBgYWKT3WZDng4iIzJhKBeyfplsDnhr/8l+U86v1dGDAZrG+ecBmoM2M4r3/7Jz6TqwR1ji3SrQw0shQAnsmA19UABZ5AdvHAFe2iLWsWalUYj3rkurAj82Au4eK5/xNcXATM8mNR4u1u7nxGwAM2gbUHypS2z2a6F+vsMn/bGVW+6aKGVSoRYD7x6jsv+QGHQDWdQbO/AAcmAb8OiD72w2/CiQYBNaXN4nAPSeGKepya9PPk6m+6lJF/lLpX0U1uuhfdvAAymczM3/SoG/2udVA+gvT++Ym4gawsrloHXdimRhsvJaP9njlvEVxNw2FrZjRz+rZXf0BnHLe+mvEy1Uz/VirNNcPBn06ATW6ip/vHgS+8QX+GAl83whOd37T7ZcSD9w7qH9bmur6hgNnWmrRynHw70DA/4lii1nrKHRZLGpjeLfTFU60cQbafyZ+luShNZ5hxlNpYeUoqtrf+FNcHvgL0Phd0/vW7Ab4Gny+3Nypfzk1Abj3r/h83/cx8EVF8Xfg/JrCP/dSothS3Nu2bYv9+/cjIiIC06dPh1wuh5ubGx4/foxNmzYhLS0NH330EZ48eQJHR0csX74cK1euxO3bt7Ft2zYEBARke9ykSZMQHh6OGjVqYO7cufj+++/x8OFDxMbGIj4+HoMHD8Y///yDkJAQLFy4EPXr18/2PG1tbTFgwAAcOHAACQkJ2Lp1K5YtWwZAzNafPHlSex5PnjxBWloaunbtiiNHjiA8PBwrVqxAeHg4Vq1aBYVCgYiICAwcOBBnzpzB7du38c4772Dw4MHa5yMmJgazZ89GamoqLC0tMX/+fJQtWxYTJ05EYmIiUlJSMHXqVLz55pvF9VIREVFBqNKNWz7FhxX/edTqLv6VhHv/ipmp5BjxxVdTFf15uPG+MSG6XuaXfhYp4wCQnCrWXF/bBpz+QfSR1qx/vbhO1ys58aloTzXpJmBbrmgfV2Gp0Vn8A0SqbcRVse5YKhdFstZ3FRWjGwwFWs/IX5py2AX9yzHBYnbQVEr0+TXQSz1+cFwMmJjqLe/oKQLmrKnFT28AazuItaXZZQ60mSXacr2IhVoihaTdbNOFzjp8JmYStYMAEqDdnOxTuUuDpzfFc185IO895w0F/J9Io761U7wGbWdnf1sSg/k2iQQ5tkzLzoOTIgvDsDf2mRX5q6Tee5UoTBgXKoLo23uAOwf099FUAQeA82tFFgAg2nIN+V1UUr+1UzcLXb6eKKZWs6tYGy5V6PfRPjw/y5IeNVxurAK6zRCfHQobUUE9a2FHh8x2Z+3mAk5VRV2JrOnsErlow1imElC9o/FjTIkD6vYRA29Jz8QAZEU/3e9DnbfFZ1VMDoXblAUcRClREvHY7x8W/yAF6r4NdFsisgYMB/OqtTeuX2CqdoajhyiIqemn/iIV2PuRGJBxLWD3gFKs2NusLVq0CO+99x42bdoEf39d5cXk5GRMmjQJW7ZsQWJiIgIDA/Hee+8hICAAAwYMyPa4lJQUTJkyBVu3bkVcXJx2xt3Kygpr165Fx44dcezYMaxcuRJjxozB3r17cz3HcuXKITY2m5SXTG5ubli3bh28vLwQFhaG1atXo2PHjtr7j4iIwHfffYe5c+fixx9/xKJFi7B69Wps26af8rVw4UIMGzYMmzZtwujRo7FkyRKEhoYiOjoaK1euxNKlS5GSUowtNoiIqGDklqIKcFaeTUzv+ypKjAS2DhZfnuMeii/MV7eK62qaqG7+23Dg71niZ8PgUuPpdbE+W8NwyYDyhVjXXhpVqAd8eAMYsRcYc0ysvY64Jr7gHlsI/LcE+Ot9UVwu6EDut2f4XnOpmX2Qa6oXenb90e1cRBpr1vRrQLzOt02kp2u4+YvHN2wH7nX7E2jyvun9nH3EfmOPiyr7k28VrLK+udjxHvBjU2BDN9FHO6fWYjmRSoEWk0VV9QGbAJfq2e/b3GBpacC4grWxOr/aODgHdAXWIm6ILI3csickErFevOEIoExF0W3AxmAQLeSYqMsQ/1hkf2gC9rBzYkCgfG1g3EmgzSdAt6XAqP26AYpKDfSDc8CoxoNU+UK3TlymALosFOu+AZGp0iEzxV0qBRqNFMXfGo0GrMuJlOz+G0yn3ic8Ada0BxZVBZbVBu78LQYIPd7UPefKNDE7PuaYmG1+pRhk5fwxUrzn01OA+kOMd392H9gyGPjSXaSyPwsGGo0CKmsmHiVAw5Hi+Xt80fh4U9teA8VeJC44OBgNGogCJw0bNtSuMXdwcIC7u1i34ezsjBcvXuTpuEqVKsHNTYyCNWjQACEhIQCA2rVrAwDs7e1RrVo17X2kpuaeTvLkyRNUqGDcVzNrwXvN7ZcpUwZeXl7anzXryn18fKBQKGBvbw8PDw9YWFiYvP87d+7gp59+wpo1a6BWq6FQKODj44MhQ4Zg8uTJUCqVGDZsWK7nTEREJUylMi50FJvNmtFX0cNTuvR+jeAjgN9A0V7IkCoNOP29KNjk2RS4+qvp280aDFRqIGbVNCQyURW4tFJYibTdwD3Gz91/i3Uzgrf3Au/sNN2iSaPLIvEl+f4RUYm65/Ls923+och20FShrj80+/7XAND4f0BCBHB8sf72rLOgpljaAd5tocxt2aBUWvyvY3qKKG71+JJ4DRr/z3Tfd1POrRZrZx0qA62n6dL0wy6IWUCNZ3fFjGDbIl7qUn8Q4FJDBL0VfEUl+IKQmVh6KZEBzScDO98XrQ4B8V4ZeUAE33lh5wKM3C8qsGve0yH/AUe+FG3sDHuKayqcl/UCWmVTr8KQ/zu6deUAEjzawyFrkU7f/iIdPfqOmOm2sDF4nBKg+9finyG1WsyEl6kE/DNbNyiY+FQsx5lyRzeIcf0PYN8UkUHh0QSwdRYzzq+CctX0lyoBANTiPV/WSwzInFmhW0svsxTZNvePiMsPjoulN2OPAaP/Ea+zpb1uRt2zqS6TChCZIR4BRf2ozFKxB+jVq1fH5cuX0apVK1y9elW7XWIijUsqlUKVWTAmu+MiIiIQGRkJV1dXXLp0CX369MG1a9dM3l5eJCYm4vfff8e3336LhIQEREWJkc/Hjx8jPl5XrCK328/r/Xt5eWHUqFHw9/dHcHAwzp8/j6CgICQlJWHVqlWIjIzEwIED0aZNmwI9HiIiKiaqdJGunJWp1O5XVYV6MGqppAm6InMI0O78LYLLmGDgwnoR9Gl6EJf11l+L++Z7orL19T/ELFjH+aZnuUqbCvX0e5wDBhX41eIxe7XO/jZsnYFB2QxyGHJrCEy4JNbtOnqI/s258R8KnF0JpGUW6rJxFlXFDUXfFQXo3BuLAP3QXFR9dAV40hNoNT1/7bqK0u6JwLXMDI/AXWJwrePnuR93YZ0IwDRCTwPjL4nZ3ecRxvsbbgvcA9zcLoL7puPF61YY3PxN9wTPjybvi/PTDNy41gY6LRAtw8LO6vaLfSACsY7zTd6MSRlpxl0lnlwSgxeWZfQHewoywNByinhO7x8FKvoi3K4pHIL2i2Uy6Zmz6ekvRFp65Tf0j02OESn4tq5i5j9oH3B8iRgcrNNbDB4+uyf6wasMBhNSMrMAynmJte67xuvS8kNPi0GB2Af6leJLI6cqwPA9YlDr5HIYzaQ/OisGMpyqiOyB8nVEXYEtBuvPw6+IopQWtmKfrOr1Fc/z+bXis6PNLFGf4DVUJAH6yZMn0bt3b+3lpUuXan+eMmUKZs6ciXXr1sHe3h5yefan4OHhgTt37mDDhg3ZHufo6IjPP/8cT58+RYMGDdCqVStcu3YtX+d75swZDBs2DFKpFBkZGRg/fjy8vLygVCphb2+Pfv36wdvbWzvDX5imTZuGuXPnIjU1FSkpKZg1axaqVKmCH374ATt37oRCocCECaU41YuI6HUhtwRq9QBu/aXbVrdvyZ1PcSvnLQLtw5+LIK5ObzErCQA+HfRnvg1JpUD7ueJf3CPg+m+iuJTfQP3Ua4WVWN/a8zuxBlVa7Cv1ioaTJ9B9maianV1V6TJuhXuf9hVEu7i8cqoCjDki6gVIFSIt2DC4vPIrsPP/IL68S8TgScJjWAGixZVKqd8yLvgwcG6NeF2bjhcZEsUhIx248Yf+tqtbRYAeelb8Djt6iOfHwqDaftbfb0As5wi/KvpYe7cB7MqLmVUAgES/QNb1P0TPbY27B0Uad2G2xXsZFf2A8RdExkYZNzEYtsnEIAwglrTkh3N1EQBnrdNRpbkoHDjkD+DfeUBiBOA7UKQ851dMiBgQrdoSqPM2LM/tBw6N0h/0AkRbPysHoMn/ictRd4B1HXW/d1VaiH7dmgD06Q3dsSnxMLm2/68PACcPoM5b+pXbAXF5/CVRMDRwV/4fl7mIeySeo+pdRHHOyJv616clAUe/1F22dQEqNwbcGukvU3KpZfw7lVXr6eLfa06iVhdvHftdu3bBz88Pnp6e+P3333Hp0iV89dVXRXbc6yKvje8pZ3weqbjxPfeKSU0UlY+fXAaqthAzCIZrd4tSeopIK1crRXVjEym7Rf6eU6aJ2SrDHvAnvhHV2w2LCA3+TaS5Foe0JFEIy7Gy6YJoRSX0jJh9lkhF/2X3hqb3e3pTrF825FoHGL67ZIvhaWYOc0oD/7oOkJBDYUTn6sAHmenBjy+Jtbya9GaFLTD+Yt7Tpl+GWi06AWQNFl3rAO0/zaxon/nV2LM5MNKgdtHuicDFDbrLUjkw6Zauan1MiFi6kRwjiv1lnQ3+pY9oLZXVmKPZD0yoVJlrtWPFIJepIntFJS0J+Kqycfo5AEACDNsuZpvz49F50TUgJkQMZnZZmH3tg9RE8RmiVosBEGsnMZBlypPLwLouuqJrns0QbVUFzkGbTe/vUFkE7o4eYm18TrUU8kNuDVg6AElZsiZaTQNafizObdvQzE4LBiRS8ZaTysTzbTioYC5klgAkQIaJ2liGRfgAYGqwyFrYPgYIPSWW3rz1Y+lelpSDvPxtzevf32JPca9YsSImTZoEa2trSKVSfPnll7kf9BLHERERFRtLO6Dd7JK57/jHwOrWupktubUI6io3Lt7zkFuYTmNu/qFoS7ZvKnBxPQCJaFHmY6JCclGIugNs6KpbhtDkA6DTF8Vwv0HAzz10Batu7wPeP6Nb861S6TIBylUTqePJ0brja3YH+m8q2WyBw58Dp1eImd6mE8S6a1PSc+inDuj3Q768UT/4S08SacaNCjB7ml8SiXjtd44Ts/pyazGzf+4n6KXuPjwh2kmVr63b1vJjMeASdVus2W43R7+lXNmqoqiZKbYuhiciXm9T1Grg1366gN6+IjD6oBhcKg7PI0wH51aOQN91+Q/OAfE9JvnWAAAgAElEQVRZ9G427ZNTn4uMitQE4NQKIOqWSLFXZWTWsJCIvuRdFxsfe261fkX0hyfhjJPZn4emVkjCY13huDyTAsgmgFa+EAXxkiKB2BBx28cWizaTvgNFy7a4h2KQ8Pxq3ey8WiVe3w9viM/wiOv5PKdiYrhEISvDGhqQiOyYNtNFgb8MZcE7GryGiv2Zaty4MbZv315sxxEREb0WTizTTztVvgC2vwtMvFJy52RIIhHteNrNEZez9sWOfyxm155cEWmmnb807gt+/6ioJO3VKnPNez4cX6pfI+D09yLjICUuM6W5svjyX619gR5atm7t0gXngHhdAveIwYndH4pUa7vyQOcFQO2eIvjZ86GYZazeOTOdvwSD87sHRcE6jaNfiorLXq2N9200Wqzd1ajRBXhwCkiNF4MPDd4Rqb7Rd43bbgFiZjO/AneLgMfNXywpyetz5dtfvM8irgPujUTF+8ubjPcznOF1cAP+74zIdrCvkL815C0+EkGoJgW+6QfZB9wPT+nPtj8PFxko+Vn3/TLKeolU8BRd/SVIZKJYYWEvRbh7ENgyyLiCfOStLBfU4vHXHwJUyr5dcrY0rQItHcT7UcMw6FTYZj/QZFgnwhSZQmQXnPoe+CezS0V6smgR+fCkGBCKCTY+7nm42J7fpQPmIj0ZkFtlCdTVwLGvxGeFdxsG5/nEZ4uIiOhVEHraeFvsg2I/jTzJGphr/Dla9xiu/iqC2r5ZKvoemisGIQAAEqDPGlFUKK+yzkprXFij+znmPvD7SGByoHF6/stwMLF2vEwlMaumKVKW8FgMpng2E4MPEy6LmcO8VhUvSo8vmd7m1dp4e9tPxNKB0NOin7VvfyD9Be5dOY5qlSsBa9ubmGnLVLdP/mdljy8Va5e153VRpE3nVZmK+in1zSaKoFhThbr+EDEjbkgiMW7zlRfOPsCEKyL4dqwsKq9nJ/V53rYVFYlEVGrfNhSIDxPp4EP/zD7N/GXsnmC6vZsp8Y+MA/Q3xoiq+obrvzV6/SAGb1KfAye+FgXusqN3HhIxUBETLGa4a3QRRQJz0mCo+P/mDuPrTHWz0LAuK/rduzcuvJT74maqEF7YBRGgU768ItVViIiIXnO2rsbbCjLTVBLSXxgPMARnSYVNSxYp1lrqLMF6HtUfnPs+qQkidbkw1e0DVG2lu+zTUay/Nez9rkzRL0hlDsE5AFRpZmJbC9P7SiRi0KTbUsBvgLhsYYN0ew/R5z274NzRU2QO5DdT4Nwa/csX1otU2oJy8xcFvXp+D7yzSwR2hc3CBvBpn3NwDoigxinL4IBUkb+ifoWhfG1R7X92pCgeVxTBuVoNPH+a+36AWC/u1dp4e6X6wP+dFssUGo7Qv05hK+pxKKxEu7dWH+uyZGycYRQKZc12gVq0b5sRBky6KYpYlqumu9q5hlh+UtEPqOAH9F0vZoyB/HUr0Lz/FVYiiycvXRV8B4puDPn2EgUJsz52U0zVSPA0UVODcsUZdCIioldBwHvA/SxBrcwC6P9LyZ1PfiisjXvs6s1Qqo1TS/PbtqhuHyA9FfhLU2XcBEuHwi8eJ7cEhu8Sha4kMt3j8myqX91YYSu+6JubKs2BLouBU8tFwN18UsHqGhj2nc7K/52CnZth+rnCSqQhvwz78sUfCJsitxRrzi+sE0Xi/AaWngG3/JBIxNIOwxlnS3sxsOXTCbi9W8wwN5+UfaE8pyoiAwIAnKoi9ew6WJZ1F626bMrq9rN2EpkAqc8BhY3oa34mcyBGphAV/vXOT6q7TysHYOxx0YYNAGp2E+/B2iYq3fuPFJkSWVmXA6DWFVOzcRZFArMucyhTSdQOeXAC2NDN9GMFRGHNGt8C59eIz5HHl3UtEHNStYVYSnL9j5zXlBuyKQe8e0Ssnb+6VTxPVg6ibRoAQCIGSNKSgZPfiE3NJpoe4KNcFXsVdyoarARdOPg8UnHje44K1c0dwOXN4stUi8kmZ+nM9j336JxIc48LFcWh+v0MuGQpKrZvqliDqtHzu4IFdpc2An9/ItaherURa4hv7xUpvF0W5txrvDBlpAMHZoiWcvaVgE6fF/76dzMRGBiIWpXKAGvaibW2gAgSKvqJ6uT+wwvWauza78COMbrBm05f6dpnUemRlgT8PUsEvha2QOsZYnnES8jX59yjc2Jw0L0x8Gt/sdwFEGntY4+LmfeCOLpABLSqDDHY0PkrkTFwZbP4v8EQwCGHFs7X/xB93ONCRVcATXG6Wj2Avhv013WrVGLw8eqW7G/PqQowaKv+IKRKJc4x+o4IuE8sMx4MdakFDNthusNCTIh4/twaAs65zLC/4gqzijsD9FeE2X7hKmX4PFJx43uOiptZv+dUKjG7ZKrwllot+ghHXBdrlV8mdVKZKoKCrDNrVGS077nU50DQAVGDoFr7wknjj7ojim+5+ZtnBgKViAJ/zqXEAze2iwydun3M6zPiWWZxuXLepq9Xq4E7f4tg26cDUM5HbFcpRR2Bsl65LyN5FiyWG1VqIIpXSqTm9RyYsVLdZo2IiIjIJKk0+6rYEglQu5f497LkluIfFS9Le8C3X+Hepkt1/UwLopdh5VA8rf4KIrvAXEMiAWp0Fv+yksnzPrtdzjv3+6EixyJxRERERERERGaAAToRERERERGRGWCATkRERERERGQGGKATERERERERmQEG6ERERERERERmgAE6ERERERERkRlggE5ERERERERkBhigExEREREREZkBBuhEREREREREZoABOhEREREREZEZYIBOREREREREZAYYoBMRERERERGZAQboRERERERERGaAAToRERERERGRGWCATkRERERERGQGGKATERERERERmQEG6ERERERERERmgAE6ERERERERkRlggE5ERERERERkBhigExEREREREZkBBuhEREREREREZoABOhEREREREZEZkOd05eTJkyGRSExet3Tp0iI5ISIiIiIiIqLXUY4B+sCBA4vrPIiIiIiIiIheazkG6G+88QYAIDExEatXr0ZUVBRat26NGjVqFMvJEREREREREb0u8rQGfebMmahcuTIePHgAZ2dnzJo1q6jPi4iIiIiIiOi1kqcAPS4uDn379oVcLoe/vz/UanVRnxcRERERERHRayXPVdyDg4MBABEREZBKWfydiIiIiIiIqDDlKdKeNWsWZs6ciVu3bmHChAmYPn16UZ8XERERERER0WslxyJxGjVq1MC2bduK+lyIiIiIiIiIXls5Buht27bV64Mul8uhVCphYfH/7N11YFPX2wfwb6TuQkuNltLi7l5kY/h4GTZsjG0MJgwYNtiQbbht+PgBw3X4kOEOw52WektLNfXGk/ePjkCa1KCFAN/PX9ybc+59EkLpc885zzHH4cOHyzw4IiIiIiIiondFoQn6kSNHoNVqMX36dPTr1w+1a9fGgwcPsGXLllcVHxEREREREdE7odAE3dzcHAAQGxuL2rVrAwCqV6+OyMjIso+MiIiIiIiI6B1SrDXodnZ2+O2331C7dm3cvHkTXl5eZR0XERERERER0TulWFXc58+fj3LlyuHcuXNwc3PDrFmzyjouIiIiIiIiondKsRJ0CwsLAIBWq4VGo4FarS7ToIiIiIiIiIjeNcVK0H/66SfExsaiZcuWiIuLw48//ljWcRERERERERG9U4q1Bj06OhqbN28GALz33nvo169fmQZFRERERERE9K4p1gi6XC6HVCoFAEilUk5xJyIiIiIiIiplxRpB/+STT9CjRw8EBAQgLCwMI0eOLOu4iIiIiIiIiN4phSboP/zwg+7PFStWhEqlQsWKFXH+/Hl06dKlzIMjIiIiIiIielcUmqDfu3cPMpkM3bt3R5cuXaDVal9VXERERERERETvlELXoB84cADLli2DXC7HqlWrcOvWLVSoUAGtWrV6VfERERERERERvROKXINeuXJljB07FgBw9epVLFiwAAkJCdixY0eZB0dERERERET0rihWkbjs7GwcO3YMf//9N6RSKbp3717WcRERERERERG9UwpN0A8fPoyDBw8iPj4eHTp0wPTp0+Ht7f2qYiMiIiIiIiJ6ZxSaoI8ePRr+/v6oWrUqHj16hEWLFuleW7BgQZkHR0RERERERPSuKDRB37Bhw6uKg4iIiIiIiOidVmiC3rhx41cVBxEREREREdE7rdBt1oiIiIiIiIjo1WCCTkRERERERGQCmKATERERERERmQAm6EREREREREQmgAk6ERERERERkQlggk5ERERERERkApigExEREREREZkAJuhEREREREREJoAJOhEREREREZEJYIJOREREREREZAKYoBMRERERERGZACboRERERERERCaACToRERERERGRCWCCTkRERERERGQCmKATERERERERmQAm6EREREREREQmgAk6ERERERERkQlggk5ERET0ltFoNZDIJK87DHpDPEp7hMScxNcdhskJkYTg74i/kZSbVOK+mYpMZMgzyiCqoklVUuQqcwtt8yT7CZ5kP3lFEVFJiF93AERERERUem4l3cLEcxMRlx2HAMcALGyzEBUdKr7usHQeZz3G9EvTcSf5Dhq4N8C05tPgZu1W6vc5EXMC86/OR6osFd38u2Fik4kwE5qV+n2e0mg1uJdyDw4WDvC194VGq8HGBxtxIuYEfOx88HXdr+Fp61nkdaQqKR6lPYK/gz/szO3KLF4AyJBnYPix4biXeg9CgRCDqw/G9w2/L9N7vilW312N32/8DgAQC8XoGdATg2sMhq+9b6H9tFotZl2ZhZ0hO6GFFj0CemBKsykQCvTHRRNyErDw2kKEpoeihWcLfFv/W1iILPTahEhCYC4yR0xmDM48PoPKjpXRtVJX2JrbFnj/xTcWY8ODDVBqlLA1y2tnZ2YHdxt3NPFogo8CP8Lcq3NxNPooAMDRwhG/t/0dnraeOPv4LMxF5th4fyPCM8LhaeuJRW0WoYpzlULfs1KthJmo7P5tvWsEWq1W+7qDoJf38OFDVKtW7XWH8cbj50ivGr9z9KrxO/f267K7C2KyYnTHTTyaYHWH1a8tnvzfucGHB+Nm0k3dcSuvVlj+3vJSvWeqNBUd/uoAhUahOze24Vh8UuOTUr3PU2myNHx29DOEpoUCAPpU7gMfOx8suL5A18bfwR97P9wLgUBQ4HVuJN7AyFMjkSHPgJXYCrNazUL7Cu0LbH88+jhW3VkFqUqKAdUGoF/VfiWKe9mtZVh5e6XeuV3dd6GyU+USXcfUvOzPuVxlLoK2B0GmlumdFwvFWNZ+GezN7XHlyRU082yGai769zn7+Cy+PvG13rmFbRbifd/39c512tUJj7Mf6447V+yMLv5d0Lh8Y2i0Gnx14itcT7xuEJubtRtmtJiB2VdmIyojCpWdK6O+e31kKbLwSPIIwWnBJX6/ZkIzQAsotUqD16zEVrjw8QWjD7fSZemYdH4Szsedh5etF6Y0m4Jmns1KfP+3QXG+c8X9XnIEnYiI3gqZikw8yX6CAMcAiISi1x0O0WshVUn1knMAuJN85zVFY0itUesl5wBwLfFaqV3/VtItHIo8hBxljl5yDgA3k26WOEH/J+ofXIi7AE9bT3xc9WM4WDgYbbfp4SZdcg4AOx7tQKBjoF6biIwIhKeHI8ApoMD7zb06VzctWqqSYua/M9HOp53RpD4qIwpjz4yFWqsGAMz4dwa87bwhVUmxJ3QPnCyd8Fmtz+Dv4I8cZQ6iM6MR4BgAc5G57hpxWXEG143LiiuTBD02Kxb3U+5jd+huxGTFoH2F9hhVf5RJjrwqNUrI1XKD8yqNCj+c+0G3fGTRjUUYUWcEAhwDcDz6OLztvCEWGqZXe0L34Hj0cdRwqYEGbg0w7PgwZCoy9docijyEQ5GH4GjuiHYV2hlNzgEgKTcJXxz7Qnf8UPIQDyUPX+btQqkxTMyfkqqkuJZwTZd4n3t8DsGSYDTxaIK9YXtxLu4cAOBx9mOMPzseJ3qfgFgohgCCQh9GUcGYoBMRkUlSqpXIUGTA1cq1yLb7wvbh18u/QqaWwcvWC8vfWw5/B/9XECWRaTE2MTL/1NrXQalR4q9HfyFLkYUqzlUQIgnRvVbbtXap3OPyk8v48tiX0Gg1AAABBNDi2edRt1zdYl8rQ56BIUeGICw9THfuePRx7Oi2w+jnaWz9trFkfv2D9ahgXwGnYk5BrVEjyCcIA6sPhL25PQDgSY7+muAUaQqUGqVeUv3UlYQruuT8qZ0hO3Ey9qTu+HzceUxsNBHTL09HjjIHzpbOWNJuCWqXy/vM3/d9HwciDujaO1o4orFHY6Ofycv45dIv2PFoh965DQ824EnOE/za4ldYm1mX+j1fhoOFAzr4dcA/Uf8YvJa/tsOqO6v0/h4CHAMgFAh130MAuiT2UOShIu+drkjH7rDdLxp6mXj6/Vx4fSH+vPdn3smbgIeNh167dHk65l6diwPhByAUCPFZrc/wea3PX3W4b7zX/xObiIgon+PRx9F+Z3u03dEW/Q/2L7RAT64yF7OuzNJNRYzLjsPiG4tfVahEJkUkFMFSZKl3Lv8v0a+aUq1Eux3tMPPfmVhycwnC08NRyaESAKBOuTqY2nxqqdxn16NdekmRFlq4WrnCUmSJ3pV7Y0D1AXnntVqcjDmJpTeX4mrCVaPXWnN3jV5yDgAhaSG4m3LXaPtOFTvpHTtbOmNyk8nwsfPRO783bC8W31iMuyl38UDyACtur8CIYyN0r3fw7aDXvo13G6PJOQCjo9z5k0eJTIIZ/85AjjJHdzz36lzd620rtMWcVnPQ1KMpOvl1wtoP1sLGzMbo/V7U3eS7Bsn5U8eij6Hrnq6IyzYcyX/dZraciYmNJ8LL1kt3TgDDEeH8D0nC0sMwruE41HKthRouNcq07sGr0MqrFWq41oBcLceWh1v0Xnv6vXrK0cIR20O2I1eVi2xlNn6/8bvevzGtVouL8RexP3w/0mXpryT+NxFH0ImIyKRIVVJMuTAFWcosAMDdlLv4/cbvmNFyhtH2afI0g18SYrNiyzxOIlNkIbLA0JpDsfx23ppuoUCIEXVGFNGrbO1N2It0+bNfxlUaFTxsPPBX97+MTgd+UcYSy5XvrTQocLXg2gKsf7AeAPDHnT/wU9Of0KdKH7024RnhRu9RUNG2Fl4t8Hvb37E3bC8cLRzxac1PUdGhIiY0moBvTn5TaNx3Uu4gRBKCKs5VML7ReLhYueBqwlXUcKmBYbWHFdivrltdfFXnK6y9txYqjQofBnwIRwtH3Eq+pdcuQ6FfSTx/MtzZvzM6+3cuNMaXkX9WQH7J0mRserAJExpPKLCNUq1EijQF5W3Kv9S06RRpCsyEZgUuVVCoFTgXdw5igRjNvZpjQLUB6FOlD1bdXoWIjAh09e+K7898rzcl3NnCGRL5swcjIoEIHSt2xMDqAwEAH/z1AeJz4l845tepqlPVQutD2JnboV2FdjgZcxJ+Dn6o5FAJe8L26LW5m3IXjco3AgCMOT0Gx2OOA8gbld/YaSP8HTnbLT8m6EREZFISchJ0yflTz6/tzM/L1gvVnKvprcF7z/e9MouPyNSNqDsCzTybIVgSjMYejV/7co80RZrhOXlaqSbnAPBJjU9wIuaE7mFAR7+OBsm5Qq3A1uCteucW31iMW0m30L5Ce7T3zSvIFuQdhLOPz+q161KxC8wEZgVWrG5XoR3aVWind66Ga94IamFrfAUQ6BJ/M5EZhtcZjuF1hhfrPY+oOwKf1vwUGq0G1mbWSJOl4XzceYSkhUAAAfpX64+YzBjdFGsABsXKysKRqCPYeH8jBAKBbu1+YVuOZSmyCnztQtwFTDo/CRKZBP4O/vit7W8l3pVAqVZiwrkJOB59HCKBCAOrD8T3Db9HbFYsRAIRPG09kaXIwsBDAxGREQEAqO5SHRs6bcBP53/C4ajDAPLqGMxrPQ+r765GYm4iWnq1xMdVP8aXx75Emjzve/5Zrc/0lmaNbjgaP5z9ASqtqkQxv25CCNGtUjfdsYXIArVca+nVjMhV5WJas2kYXmc4Jp2bZJCcA0AD9wYAgGBJsC45B/Lqxmx4sAHTmk8ruzfxhmKCTkREJsXX3hc+dj56o+AtvVoW2mdp+6VYdmsZItIjEOQThE9rfFrWYRKZtLpudVHXrfhrrstSD48eOJF8Qm89eFlUU6/oUBEHex7EhbgLcLVy1Y3aPbXr0S4subnEoHhchiIDByIO4EDEAfzc/Gf8X+D/oXfl3shWZuNA+AGIhWK082mHfeH70HlPZzhbOmNO6zkIcAzAmdgzEAlEaFC+gcF0dgBwtXLF7FazMf/afKRKU9GofCNcS7gGueZZAbK+VfoWa/u1gliKny1pcLRwRAe/DkiXp8PRwhFtfNqgmnM1rLi9Ag9SH6CJRxN8UeuLQq728u4k38H4M+N1f993U+6igVsDhKSFwMHCAV/W/hJLbi5BYm7eun2hQIj/C/w/o9dSa9SYcmGKbup+REYE5l6dixXvrShRTHvD9+JY9DEAgEqrwrr763An+Q5uJN0AAHTx74KaLjV1yTkAPEh9gHX31umScyBvtH9X6C5s7boVMZkxyFXlIiI9AjVdawLI+1438Wiid++Ofh1R2bEyPtz3YYlift000GDetXn4O+JvzG49G/4O/npLSIC83QvC0sOw4NoC3WcJAGKBGK7Wrvi85ueoU64OABjdlz1XVfhe7e+q15Kg//vvv9i2bRsWLVqkd3706NGYM2cOzM2Nr7Vp0aIFLly4oHcuPT0d586dQ7du3Yz2KahfSTx+/Bjdu3dHjRo1dOeaNGmCb74pfMrS8+Lj4xEcHIx27doV3ZiI6B0mFAixtP1SLLi2ANGZ0Wjr07bIKbpu1m6Y3nz6K4qQiErCzdINq95fhXnX5kGmkuHTmp8arNkuLfbm9kavHZEegemXpus9JDBmb9he/F/g/0EgEGBozaEYWnMoAOCTw5/opoZLZBJMPj8ZOYoc5KieLa953/d9jKk/Br/d/A2haaFo7tkc39X/Dh38OqCD37O15ZmKTFxPuI40eRoqO1XWJXfFkSZLw92Uu6jmXA3lrMsZvL4vfB+W3FwCAEjMTcS3J77F0V5HMbHxxGLf42Wdjzuv9zlrtBpcTcxbh5ypyMScq3P0Rsw1Wg2W3VqGpe2WGhSLy1JkIUmqX4MkPN348oPCRKRHGJx7PqE8GHHQaL/8a/qBvGKEk85N0iuu91RUZhT299ivmx2i1qjxIPWBwdrtN8lDyUOMODYCB3sehK+9r97nZiGyQHmb8riTor9ThEqrwoEeB/QeHtV1q4uqzlURLMnbBk4kEKF35d6v5k28YUxqBD1/wl4cISEhOHnyZKEJemkICAjAxo0bX7j/5cuXERERwQSdiKgY/B38saz9stcdBhGVkqaeTbGr+67Xdv/bybcNkvPqLtURkhoCNZ4V+SpobXJkRqTesbHClceij+FB6gNdIh+REQGNVoMfmvyg187e3B5tK7Qt8Xs4+/gsxpweA7laDrFQjF9a/IKu/l312lyKv6R3LFPLcCPpRqF7qZe2otYUG5vOfjXhKnaH7tat237K0dIRtVxr6RXna+3dusQxBfkEYdPDTbrj/FXWAcDLxgs2Zja6miZOFnnb1O0I2aE3PV2pURpNzoG8+ienYk+hmUczZCuz8fnRzxGdGV3ieE1NfE48QtNC8VXdr3Ar+RYiMyIhEojwUeBHcLBwQHWX6gYFF/eH79er7SAUCLH2g7X469FfSJYmo3PFziV6OPUuMakEvV27djh8+DASEhIwceJEiMVieHl5IS4uDhs3boRCocD333+P+Ph4ODo6YvHixVi5ciWCg4Oxfft2NG3atMB+o0ePxpMnT1ClShVMmzYNS5cuRXR0NNLS0pCRkYH+/fvj6NGjiIyMxJw5c1C3btHTwtRqNaZMmYKEhASkpaWhdevWGDVqFKKiovDjjz9CqVTC0tISCxYswKpVqyCTyVCvXj14eHjgl19+gUgkgoWFBX755RdoNBqMGDECjo6OaN26NaytrbF3714IhULUr18fEyYUXDiDiIiIiApWz62ewbZrg6sPRnh6OP53938A8gpefVnnS4O+qdJUg0rdLpYuSJWlGrTNX4Dt7OOzBgl6cVxNuIp/ov6Bl60X+lTpAxszGyy8tlC3N7dKo8KCawvQpWIXvaJpVZ2r6m3lJYCgTPY0B/IS1SU3l+BkzEl423nj+wbfI9ApEO9XeB89Anpgf/h+XdXz/J+fMQVVcl/YZiEWXFuAkLQQ3ayEkmrq0RQzWs7A1odbYWVmhaScJERnPUuchQIhugd0R9dKXbE7dHfe6G6V3nCzdkOvyr2wLWRbse815vQYWIgsUNW56luRnAOAmdAMlmJLuFi6QK7K+w6qtWpsCd6CVt6t0MC9gUGCfjv5tkHxRTtzO3xak0vQimJSCfpTc+fOxfDhwxEUFIQdO3YgLi7vH2xubi5Gjx4Nb29vDBo0CA8fPsTw4cOxbds29O3bF19//bXRfjKZDGPHjoWXlxe+++47nDyZtz+kpaUl1qxZg1WrVuHMmTNYuXIldu3ahYMHDxok6GFhYRg0aJDueP78+VAqlahbty569+4NuVyuS9DnzJmDYcOGoXXr1jh06BCCg4MxbNgwREREoH379ujZsydmzJiBatWq4fjx45g9ezbGjx+P5ORk7Nq1C+bm5vjoo4/w008/oW7dutiyZQtUKhXEYpP86yIiKnVKtRL7w/cjKjMKbX3aor57/dcdEhG9gExFJiJyIhCgCShwu6lL8ZcQkRGB5p7NS1z8q7j8HPwwvfl0zLs2D1KlFLXL1UZbn7bo4t8FnSp2QkxWDBqXb2y0Svvmh5uRqcjUOze0xlDMvz6/yCnz5awMp6EX5WTMSYw6NUp37ZMxJ7Gx80aD6dbpsnSotWqIBYhpzJkAACAASURBVM9+P/y46se4l3IPx6KPwcbMBiPrjzS6Nv6pTEUmZlyegX+f/At7C3t8VecrdKzYsVhxrr6zWrcndnRmNCLSI3Co5yHd6P6YBmMgFAgx6tQovcJixgggKLC4Z3mb8pgXNK9YMRWme6Xu6F6pO+Ky49Bxl/579LXzha+9LwDg+4bf6702vtF4PEh9YDCNuzBytRy3k2+/dMymQqlRouf+nhhQbYBBRfr9YfuN/v3Wd+P/2y/KJDO+8PBw1KtXDwDQoEEDHDiQN43EwcEB3t7eAABXV1dIpdJi9fP09ISXV94ehvXq1UNkZN40perVqwMA7OzsEBAQoLuHXC5HfsamuGdnZ+Pu3bu4fPkybG1toVDkFR2JjIzUxdG5c962Fbt379b1S0pKQrVq1QAAjRo1woIFCwAA3t7euvX3s2bNwtq1azF//nzUrVsXWm3h/wEQEb1Nxp0dhxMxJwAA6++vx9yguejoV7xfGonINPwd8TemX5wOmVqGhZELseK9FQajuXOuzNFNPRYLxPit7W8I8gl64Xsm5CTgYvxFVLCrgIblG+q9FiwJ1k2vvpF0A9MuTcPc1nMR6BSIQKfAQq+Zn6+DL3oG9sSu0GfT9vOP0ANAK+9WJX4POx/t1LvOreRbeJT2CN0qdcOGBxt05zv7dzaohG8ptsSCNguQrciGuci8wD3UASAsLQwDDg3QFepKlaViwrkJqOpcFX4OfkXGeT7+vN7xk5wniMiI0P0dO1k6AQC+qvsVPv/nc2iQN6Xc1coVvna+sDG3gVqTN7Leu0rvV5bQ2ZrZGlTW97Ev+CGGmcjMYG08APjb+0OukZvkHu6lpZxVOSRLkwHkzdrYGbLToI0WWl2bp8pbly+w8B8VTfi6AzCmcuXKuHnzJgDg9u1nT5+M7XsoFAqh0WgK7ZeQkICkpLy1Qjdu3EBgYGCB1yuJ3bt3w87ODgsWLMDQoUMhk8mg1WpRqVIl3L2bt1Zm//792Lhxo16cbm5uCA7OK5Bw9epV+Pn56d7LUzt27MD06dOxadMmPHz4UPe+iIjedgk5CbrkHMj7z3/rw62F9CAiU6NQKzDr31mQqWUA8tZsL7y+UK9NhjwD24KfTR1WaVVYc2/NC9/zWsI1dN3TFVMvTsWn/3yKWf/O0nv9+anfAHA06ihUmqK3vspfdM7F0gWNPRpjXKNxaOPdBgIIYG9uj75V++q1Mxea621TVRxpsjSDadECCGBjZoMxDcZgYuOJeN/3fYxuMBpTm00t8Dq25raFJucAsPbeWoMq2hqtBtcTrxcr1kBH/Yca1mJreNl6GbRrVL4RNnTegH5V+uGbut9gT/c9WNdpHZa1X4aV769Ez8CemH1lNupvqo8fz/8IhVphcI3S5GDhoLe/vJ2ZHb6q81WhfYy9r8XtF6Oqc9VSj8+U5P+7MFZ1XaPVGMyOae3dGkKBSaaZb4TXNoJ+4cIF9OzZU3f8dBQZAMaOHYtJkyZh7dq1sLOzK3Rqd4UKFfDo0SOsW7euwH6Ojo749ddfkZiYiHr16iEoKAh37hR/mkpBmjVrhjFjxuD69euwsrKCr68vkpKSMH78eEyZMgUrVqyApaUl5s2bh/j4eKxYsQI1atTAr7/+il9++QVarRYikQgzZ840uHaVKlXQq1cvODk5wd3dHXXq1HnpeImI3gRiodiggI+FyKLQPlEZUZh/bT7C08PRxqcNRjUYVWQfIio7mYpMg2nhcVn6I41qrVo3qvqUUl3wfuFFWX1vtW6NNgBsC9mGL2p/oduT2s3aTbdHOgC4WLlAJBAVed1W3q3wW9vfsDdsL5wtnTG05lBYia0AAEvaL0GWIgv3U+5j5KmRuj6+dr6Y2nwqytuUL9F7GHd2nN4WkwDQM7CnLkEcUG0ABlQbUKJrFqSgvceru1QvVv9v6n2D8PRw3Eq+BUcLR0xuOhk2ZjZG29YpV0e33dbzUqWp+OHcD7qt7/aF74Ofgx8+r/V5Md/FixleZzg6+HVAdEY0GpZvaHRpA5D3wOS3G7/hZtJNWImsIFXnzd7tW6Uv0mRpOPv4bIH3aO7ZHBfjL5ZJ/E9VdqqMiPSIMtlj3dHCEW2922JPuOHe5s+79OQSJjSagPnX5kOmlqGaczUMrzO81ON5lwi0Jjh3ev/+/ahTpw58fX2xc+dO3LhxA7NmzSqzfm+Dhw8f6qbN04vj50ivGr9zxs24PENXlMdMaIaV761EY4/GRttqtVp039sdUZlRunNDagwxWEdIefido1dl8OHBuJn0bAbgF7W+wMj6I/Xa5N+uanar2eji3+WF7vfJ4U/0toACgH8++ke3x/il+EsYdWoUclW5sBBZYGbLmXrbn8nVcqy9uxbXEq+hdrna+KLWF0anNhsz5MgQvZFnK7EVzvQ9o0vk85OqpNBqtXrXz1JkofnW5nrtnC2dcabvmWLFUFKnY09j5MmRuun0QoEQo+uPxpCaQ0p0nRRpChzMHWAmMl5joDAX4i5g+HH9ZK6tT1ssbre4xNd6Xmn9nPvy2Jd6SbaPrQ8cLBzwKO2R7qGCMUIIMaLuCIggQkh6CCraV0RHv45YeH0hzsYVnNSXlLOFM0Y3HI0pF6YUWQ/hRZgLzQt9nwAQ4BiAPR/uQbYiGxKZBBXsK5R6HG+C4nznivu9NMk16B4eHhg9ejSsrKwgFAqNjjCXZj8iIjItk5tOxvu+7yMqMwotvFoYnV741OPsx3rJOZC3Dy8TdKLXa1GbRVhxewXuxN9Bh8AOGFJjiEGbn1v8jOZezRGeHo7W3q1Rz63eC9/v46of6yXoQd5BuuQcAJp5NsPx3sfxMPUhAp0CdWukdbFc+hn7w/cDAK4kXEFsVizmB80v1r3TZGl6x1KVFDKVzGiC/vuN37Hh/gZotBp8VPkjTGoyCUKBEFZiK4Pq8JUcKxXr/i+ijU8brO6wGkeijsDN2g19q/Q1+EyK4+kMhfwORx7Grke7YGduh89rf44aLjUM2tRwqQFLkaVuKQQANHBvUOIYSupw5GEciz4Gb1tvDKk5BM6WzgZt5Gq5wQh4bHYsYrNjDdrmp0He3u5VnatiW5dtEAlFOBVzCsm5yUX2LQmJXII5V+bgw0ofYm/43lK9NoAik3MgbzcEIG9Zha25banH8C4yyQS9UaNGekXVyrofERGZnsYejQscNX+eu7U7HC0c9aaultW2QkRUfC5WLvix6Y+FjhqJhWKDvbxfVMeKHeFi5YKTMSfhZ++HHoE9DNrYmdsZ/blyKOKQLjl/6nj0cag1aoiERU+D7xHQQ2+NfRvvNkaT3StPrmD13dW64+0h29GwfEN09OsIsVCMn5r+hMkXJiNHmQM3KzeMbTi20PvGZuYljPXd6sNSbGnwukKtwNyrc3E06ig8bT0xofEEvYcgxf05W1Ln485j/NnxuuPLTy7jyEdHDPaZd7R0xII2CzD/2nyk5Kaga6Wu6F+tf6nH87w9oXsw5eIUvdh2dNth0M5caA4vW6+XKgIXLAnGpfhL2PBwg8Ee9aUlW5mNo9FH4WrpihRZSpncozBXE67iXNw5dPTrqDcjhV6cSSboRERExWUuMscvLX7B1ItTIZFJUNOlJkY3GP26wyKi16BR+UZoVL5Rifoo1UrMvGI469Ld2r1YyTkAfFrzU7haueJc3DlUdqqM/lWNJ5khaSEG5x5JHul2qWjv2x7NPJshNisW/o7+BW5NBwArb6/E8lvLoYUWzpbOWNNhDQKcAvTarL67GttDtgMA0uRpGHlyJI73Pl7mNTqORx/XO85WZuNS/CWjW7i19m6N1t6tyzSe5z2/pAIAHkoeIjQt1KCSv0AgwNRmUzHx3ESDLe5K4vKTy2WWnD+Vq8o1WsDtVXj6eR6LPoa52rkGRRWp5JigExHRG6+NTxsc730cGfKMAqdbEhEZk6XMQoY8Q++cAAJMajKpRNfpVqlbkVXbm3o0NSiC2cyzmV4bazNrVHGuUuh1JDIJ/rj9h27dsUQmwYrbK7CgzQK9dlcTruodp8vT8UjyCLXK1Sry/bwMY3uvF7Yf+6vkaqn/f4RYIC5wan8zz2Y41usY4rPjMfjwYKTJ04y2K0igU6BBIcS32YHwA0zQSwHr3xMR0VvBTGjG5JyISszZ0tlg3fOg6oNeaj/2ggQ6BWJO6zmo4lQFlRwqYVqzaSUe8QeAdFm6QeXuFKnh9Ob8676txdbwd/Qv8f1Kql/Vfrp9zYUCIQZWG4garoZr0F+HL+t8CRdLFwB5D2Ker/RvjLnIHH4OfljWfhksRYbLCAriauWK9R+sRzufdhBAf2vnuuXqYsV7K/BV3a8wpsEYg75tvNtA+IJpmru1O6o5V4OnjSfEgmdjsbZmpbM+PP97eV4563Klco93HUfQiYiIiOidtqjNIiy/tRyh6aFo4dkCn9b8tMzu1dGvo25K+4vyd/RHDZcauJ96X3fO2Oj98DrDEZsVi1Oxp+Bm7YbJTQreCq002ZjZYH2n9YjIiICtmS3crN3K/J7FVcmxEo58dAQ3km7A29a72FXHa5WrhVN9TuG7U9/hSsIVAHnV2qu7VEf/av3h7+CPEEkIkqRJsDO3Q1f/rrCzsEPD8g2xsM1CbAveBguxBfpX7Y8WXi0AAC29WgLIW5rx16O/YC4yx6Bqg+Bj74PL8Zcx6tQo5KhydDHYm9ujlWcrHIw6qDtnZ2aHLGXelnlCgRCTmkxCuwrtAOQt37iaeBVOFk4IcAzAwMMD8SD1AYC8hzWLghbhj7t/GOx+8FS9cvXQs3JPXE24igepD9DUoyl6V+6NL459gaTcJL227tbu+Lxm2W6P964wyW3WqOS4bU7p4OdIrxq/c/Sq8TtHrxq/c2UjTZaGP+//idjMWLzn+16h29Mp1AqYCc0gEBQ8+vk2KcvvnFqjxunHpxGTGYMgnyD4O5TdjAStVosHqQ9w5vEZaKFFj4Ae8LL1wpPsJ9gbthd+Dn5o690WZ+LOIDIjEkHeQajmUvD7VmlUuBB3ATK1DK29W+t2GZDIJBh3ZhxC00LhYuWCRu6N0MKrRYGzSDLkGfg74m9otVrUcq0FmVqGem71YC4yL5PP4U3w1m+zRkREREREBXOydDI6PdqYdzlxKm0ioQjtK7R/JfcSCASo4VrDYHmAh60HRtQdoTv+wO+DYl1PLBQbTbqdLZ2x5oM1xY7LwcIBA6oNKHZ7KhmuQSciIiIiIiIyAUzQiYiIiIiIiEwAE3QiIiIiIiIiE8AEnYiIiIiIiMgEMEEnIiIiIiIiMgFM0ImIiIiIiIhMABN0IiIiIiIiIhPABJ2IiIiIiIjIBDBBJyIiIiIiIjIBTNCJiIiIiIiITAATdCIiIiIiIiITwASdiIiIiIiIyAQwQSciIiIiIiIyAUzQiYiIiIiIiEwAE3QiIiIiIiIiE8AEnYiIiIiIiMgEMEEnIiIiIiIiMgFM0ImIiIiIiIhMABN0IiIiIiIiIhPABJ2IiIiIiIjIBDBBJyIiIiIiIjIBTNCJiIiIiIiITAATdCIiIiIiIiITwASdiIiIiIiIyAQwQSciIiIiIiIyAUzQiYiIiIiIiEwAE3QiIiIiIiIiE8AEnYiIiIiIiMgEMEEnIiIiIiIiMgFM0ImIiIiIiIhMABN0IiIiIiIiIhPABJ2IiIiIiIjIBDBBJyIiIiIiIjIBTNCJiIiIiIiITAATdCIiIiIiIiITwASdiIiIiIiIyAQwQSciIiIiIiIyAUzQiYiIiIiIiEwAE3QiIiIiIiIiE8AEnYiIiIiIiMgEMEEnIiIiIiIiMgFM0ImIiIiIiIhMABN0IiIiIiIiIhPABJ2IiIiIiIjIBDBBJyIiIiIiIjIBTNCJiIiIiIiITAATdCIiIiIiIiITwASdiIiIiIiIyAQwQSciIiIiIiIyAUzQiYiIiIiIiEwAE3QiIiIiIiIiE8AEnYiIiIiIiMgEMEEnIiIiIiIiMgFM0ImIiIiIiIhMABN0IiIiIiIiIhPABJ2IiIiIiIjIBDBBJyIiIiIiIjIBTNCJiIiIiIiITAATdCIiIiIiIiITwASdiIiIiIiIyAQwQSciIiIiIiIyAUzQiYiIiIiIiEwAE3QiIiIiIiIiE8AEnYiIiIiIiMgEMEEnIiIiIiIiMgFM0ImIiIiIiIhMABN0IiIiInrjaFWq1x0C0RtLlZwMlUSiO9ZqtS90HXVWFtSZmaUVFgEQv+4AiIiI6O2nVSiQe/06RM4usKxS+XWHQ28wlUSC2GFfQnbvHkQuLvCcPx+2zZq+7rDoLZRz9Srivh0JdUYGxOXKwWfNalgGBupe12o0UERHAyIR0rdshSolBQ7du8G2desXup8qLQ2pf6yCPCwMtq1bw6F3L2Ts2AFZcAhsWraAQ5cuxYv7ylUkzZ0LrVwGly+/hEPXrlClpCDz4EFAJEbulSvIOnYMEAph3aABZKGh0ObmwrFPH7hPnIDcf/9F4tx5UCUnw6FbN7iN/R4CsX7amLpxI1KWLYcmPR0QCuHYqxfKT5sKgZDjvy+LCToRERGVKWVCAqIHDoLy8WMAgGPv3vD45efXHNWbT6tWI23zZmSfPw/LKlXhMuwLiOzsXndYZe7xN99Cdu8eAECdmorHw4ejyvVrBglEceRevw51VhZsmjeH0Ny8tEOlN1zsl8Ohzc0FAKiSkhD72ecIPHsGACCPiEDM519AFR+v1yfz77/h88dK2AYFlfh+cd+ORO61awCAnPPnkbZzJxShoQCAjD17oEpIhMtnQwEAuTdvIn3bdggsLeE8eBAyDx6EZN16QCiEJjtbd834seMg2bQJiqjovGT6eWo1cq9c0R2mbdwI8woVkLRoke59S9atg9jdHS6fDnn2Hv85iqQZM59dR6NB+o4dsGnRAvYfdCj0PWq1WmT9cxSy+/dh3aQJbFu2KPHn9LZjgk5ERERlSvLnOl1yDgDpO3fCaeBAg5F0VUoKBObmENnbv+oQTYJWq4X0xg1Ao4FVgwZFjkQlL12K1BUrAQA5Z89BFhyMCv9b9SpCfa1k9+/rHWvlcsgePIBV7drFvoZWq8XjEV8h+/RpAICZjw/8tm6B2NW1NEOlN5gqK0uXpOrOJSXp/pw0Z65Bcv5U+t69JU7QlYlJuuT8qafJue66f/0Fl8+GQhocjOgBAwGNRnc/yOUFXlt263ax40jft9fgfacsXw5lbCzKjfoOInt7ZJ87a7RvzsWLRSboSbPnQLJ+PQAg9X//g/ukH+A8eHCx43sXcA4CERERlSlVSoqRc8m6P2sUCoR16oTQlq3wqHETxHz99asMr0yoJJISrcvUKhSIGTQY0QMGInrQYER9/DE0UmmhfTL/Pqh3nHPuHNQZGS8Ub2Hk4eHI2LcPiucesrxOxh7gmFWoUKJr5P57RZecA4AyNhZpW7e9bGj0FhHb2UGQb1aF0N5OV/tAHh5ecF+Xkj/oEdnbQWhtXUSbvO9+0oyZuuQ8L5iCk/OSkt+7D+SbjaLJykLali2In/gDAMCiUiWjfRUxMYVeW6tQIG3rVr1zkvUbXiLatxMTdCIiIipTDj0+1Ds28/SETaNGuuP48ROgjIzSHeecOImMf46+qvBKlVapRNyY7xHaoiUetWiJxHnzitUv8+gxvdEz2e07yPj770L7mLm76x0LHRwgtLIqedCFSNu6FRFduiJ+wkSEf9ARmYcPl+r1X4Trd9/pHVs3awaxo2OJrqHOP9UXeet/iZ7nMWsmYG6WdyAQQJOZhbB27ZFz6RJsgoyvMxd7esBl6KclvpfQygpuEyYYJMd6bf5L0GUPH5b4+iVhWbMmhPaGy2WyT51C7u3bkGzabLRf7qVLkGwoJOEWCID8M4O4Zt0APxEiIiIqU7atWsFn1R+w79wZToMHwXfzJr2RqeyzhtMlJX/++SpDLDUZ+/Yh89AhQKsFlEpI1qxF7vXrRfZTS1INz6VKjLR8xm3s9xA6OOQdmJnBfcIEgxG/l6HVaJD82+/PBaRG8u+LS+36LyrrqP7DG+mtW9AqFCW6hrmfr8E5kcO7ubSCCubQpQuq3r4Nq2ZN8/5NI2+ae9zYcRBYWBq0F9jbo9KRI1ClpSHpt9+QtnMnNMUc3ZZs2IjUVasKXeKTc+4cFPHx0D4/el4GZLduQZOZZfiCSITovv2giosrsK9ky5YCXxOYmUGU72GayMnpheN8WzFBJyIiojKlVamQfe48ss+dQ86ZswajP8aKc72pv7TJwwynvcpDw4rsZ/f++xDa2OiOBZaWsO/UsdA+VnXrIvDUSVTYsB6Bp0/Bsef/lTzgwmg0BtPs1TnZ+sfZOcg8ehTS28Vf4/qyci5e1DvWSqWQx5Zs+r380SODc2ojSzEKo9VqoVWrS9SnIOqMDDwePRohDRoiqm8/yIKDS+W69PIEAgGUUdF659SpqUhbu9agrdDKCrn/XkFUn75IXfkHEn6agsdff1PkPXIuXkTizJlQxsVBLSn8wZz0xk3YNG9m9DXLunUgLOFskhIpxtaGmvS8ZTZahQLK59bsA4BGKoUqIUHvnLKIafHvIiboREREVKbStmxB2saN0GRlQREdjbhRo6FKfTZi7PL1VwZ93MeNe5Uhlhrb/NNezcxgU4wqxWYeHvDdsgWOffrAoddH8NuyGea+hqO8z9Oq1Ujbug2pK1cidfWaUl9/LhCL4djrI71zTn376f4sj4hEWLt2iBv5HaL69kP8xImlev8CGUkSlKklS66t6tbNm277/Ln6DYrdP3XdOjxq0hQhDRoice68F95D+qnEuXORdfgINDk5kN6+jccjv3vpa1LpsW3ZsljtbJo1RdqmTcBzD25yzp+HPCKi0H45/14xOCdyczOcESMQwKZFczj26mX0OlZ16xlWajfG0hI+f66FoCwKcorFyDx2DKFBbRDWOgiRvXpD+V9SLrSygrm/v34oNWqUfgxvOCboREREVKZyr+lP8dYqFJDevas7dhk0CC7Dh0Po5AhR+fLwXLAAFv4VX3WYpcKmWTN4zJwJyxo1YNWgAXyWL4O5t3ex+lpWqQyPn6fD89dfYVm9epHtkxYsRNK8eci5eAmSP/9E7FelX1zPffJkePz6Cxz79IHn/Pko982zeyTOng3Nc4XwMvbugzwistRjMGBmuEbXzM3dSMOCCcRi3ZTlp9Q5OcXqK717D0mz50CTmQmtTAbJ2rXIOnKkRPfPL3/1bmVMjF7FcHq93CdOgGO/vjDz8YHYw6PAdi6ffWb0+ykQi5F77RrixnyP+AkTDWYRWdY0TFK9Fy5ApaP/QOzm9t9FBHAdMxpiJyfjtSAEAlhUDjQ8b4xMhoxduxFw8G8IiihMV6R8a8htmjbFk8k/Qv1fTQfZvXtIWrhQ97rnnNkw9/MDkJecl5865eXu/xYq8wT9woUL6N69O2QyGQAgMTER3bp1Q2JiotH2jx8/Rp8+fcoklk2bNhmcW7JkCT744AMMGjQI/fv3x9ChQ/HgwYMyuf9TycnJmDZtWpneg4iIyFRY1a2rf8LMDFbPjZpoNRpopVJoZXJAqYQm3zTqN41jz/9DxV1/wW/zJti2alVm90nfsUPvWHr9utHiZy9DIBLBsVcvePw8HQ5du+i9Jg8JMWife/NGqd7fGJtW+rMUBDY2sPAsOGkyRrJuvcG59O3Fq+Kec+Vfg3PZ586X6P75WdWuo3cs9vTglm/FkLZjB2KHj0Di3HllWuRPaGMDj2nTEHDsKHxWLNdbjvKUyM0NloGBcBn6GQQWFrrz9l26QCOVIvrTocg8dAgZ+/YhesBAKBOfPYCxe+89OA8dCoGlJYTW1ij33UhYN2wI5ZMnz3aD0Gp1267Jbt8xDFKrRfqOnRCYmRXrPcmCgyEuVw6VDh+CeeVAwMICQhcXWNSrW3Tn55gHBsKyTm0IrKxg9/57cB48SO/BHaC/zMeqVi34Hz6EyteuoeKuv2Bewh0Y3gVlvg96ixYt0LJlS8yePRuTJ0/G6NGjMXHiRLi7l+xJZ2lYsWIFBg4caHB+yJAh+PjjjwEA4eHh+Prrr7Fv3z5YPPePqzSVK1eOCToREb0znAcOgCIyAhl790Hk4gL3CeMhLldO93rG/v26fXHVUikSpk6DdYMGBW7l8zZTJiYCGg3MChmle8pYYTSNXA5RWQRmhLmfH1T5BlxexXRVjyk/4XFqCmS370Bcrhw8Zvxa4uJ4ZkaWD4hdyxlpaUgrMyz6VdSWeEVxnzgBaokEORcvwtzfHx6//gKB6FX9Tb6ZUtetQ9LsOXkHp09DeuMG/LZtLbzTS9BqtVA9eQKLSpXgf+gQkhbMR+b+A7rXzT09kbpmDew6dID/wYPIPn0aZl6esA0KQvLixYBSqWuryc1F9skTcPov/xAIBHAfPw527dvhydRpSF6yFNLbd6DVqKH9b5ATADL3H0C5b78tcDmL7NatYr8fdUYGnkybhuzL/0IVFZUXl1wOeaphwcrCOHTqCNfhw3XHWo0G5r6+UEQ/W7ef/0GlQCCAyNbwIQflKfMEHQBGjx6N/v3746uvvkLz5s3RokXeWqw7d+5g+vTpsLGxgYuLCywsLPDNN99AIpFg+PDhkEgkCAoKwtdff43Hjx9j8uTJUKlUEAgE+PHHH1G1alXs378f69evh7m5Ofz8/PDzzz/j8ePH+OGHHyAWiyESiTB37lzs3r0bGRkZmDZtWqHJcaVKlVCjRg1cv34d169fh6urKz7++GOEh4dj2rRp2LhxI7p164aGDRvi0aNHqFixIlxcXHDt2jWYm5tj1apVWLlyJaKjo5GWloaMjAz0798fR48eRWRkJObMmQNXV1eMGTMGO3bswJUrV7Bo0SKIRCL4+PgUGP/reKBBRERUGgTm5vD45ReU//lnCPKt+wWA3Cv51l9qtci9ffuNTdCVCQnI/PtvCCws4dC9G0RPK60XQqvR4MmkycjYtw/QamHXqSO85s3Lm4pdZ/ZpZQAAHPJJREFUAMvq1SF97hdyoZ0dxC4upfIeisN94gREfdxfl0DYBgXBqmrVMr+vWfnyqLh9O1RpaRDZ279QIuvU/2Okrl2rq0YtsLBA+WlTC+2TcfAgUlasgDrNcJaClZEpyiUhdnFBhTWroVWrmZgXU+YB/W0IpbduQfH4cbGXlJSEPCISj7/+GorISIicnWFVvx6yT502uL/01i0kL10G300b4TxwgO61/FsiAoA43zmtWo24seOgevIEAJB9+rRBGyBvK8fSqOKuTk5G+rbtL30d88BAaHJzoYyLg7m/PwQiEbxXrEDSvHlQRETAtn17uH5T+stv3mavZA26mZkZ+vTpg4sXL6Jnz56681OnTsXs2bOxYcMGVHhuekNubi7mzZuHrVu34ty5cwgODsbcuXMxaNAgbN68GZMnT8akSZOQlpaGJUuWYP369di6dSvs7Oywfft2XLx4ETVq1MCff/6J4cOHIyMjAyNGjICDg0OxRq5dXFyQVsg0mZycHHTt2hWbN2/GtWvXUL9+fWzevBlKpRJhYXlTOCwtLbFmzRp06NABZ86cwcqVKzFs2DAcPHhQdx2tVouffvoJS5cuxaZNm+Du7o49e/YYjZ+IiOhNZyw5BwCNVGZwTiB8M5MUZVwcIj/sgaT5C5A4YwYi+/SBJje3yH7Zp08jY+9e3brorMNHkHnkn0L7eMycAbP/fn8SOjjAc97cQhP6F6GIikL0oMEIrlcfMcOG6U3LtaxWDZWO/oPy06fD54+V8F6xvFTvXRSxk9MLJ7MCgQCBJ47Da8liuE+dgsrXrsKikKJ88rAwxI8bD0VYONT5RhjNfCvA4bnfb18Gk/Piy5+8CiwsivUw7EUkzpoFRWRefQW1RILs4yf0CsE9TyuVIi3fPuEOH36ot9THtm1b2AYF6bXJuXhRl5w/pclXF0FgZQUzT09AaPxnqcjZuXhvqBgENjbAc99HgaXhtnIAkLJ8BUJbByGiW3eEd/gA8tBQWPhXhM+K5aj0zxG4jx9nsFOHJjcXspCQEm+P+K54JQl6XFwcVq9ejXHjxmHcuHFQ//eFTkpKQmBgXjGDBg2eVc6sWrUq7OzsIBKJUKtWLURGRiI8PByNGjUCAFSrVg0JCQmIjY1FQEAAbG1tAQCNGjVCaGgoevXqBScnJ3z++efYvHkzRCX8YRcfH1/kiHWN/6Zw2dvbo9J/T/jt7e0h/2+vw+r/FXexs7NDQEAAAMDBwUH3OgBIJBIkJSVh1KhRGDRoEC5cuID4+PiXjp+IiMgUKZ88gTrbcH25yMior0D0ZtaxTd+9R2/6qTI6BlknThbZT5FvG6e8c1GF9rHw90elf47A//AhBJ47C7s2bUoabpHixk9A7tWr0EqlyDl7DglT9UeZzdzc4NS3D2yDgiAQvll/Z5knTyJp3nykLF2G1NWrC22b8++/QL5RS6t69eD8+Wfw27kT4jd0W8A3WbnvRj772SESodzoURDZ2ZXJveThRW+VWBihtTV8t26B347tqLhnN3xWLNd7GKOIisLjkd8Z9NPk+3mplUoR8WEPaIzM4gAATZaRvctfkMjGBv5798Dx436w794dlgWsTZeHhuriVMbFIXHuvEKvm3X6NEKD2iDywx4Ibd9er2Ao5Snzn6QKhQKjRo3CpEmTMGTIEHh4eGDp0qUAgPLly+tGnG8/t39meHg4cnJyoFKpcOfOHQQGBqJSpUq49l+Fy4cPH8LV1RXe3t4IDw9H7n9Ppq9cuYKKFSvixIkTaNCgAdavX4+OHTti9X8/dIuzXcWjR48QFhaGunXrwsLCAsnJyQCA+/fv67UraBSguK8DgJOTE8qXL4/ly5dj48aNGD58OJo0aVJg/ERERG8idVYWoj8ZgrC27RDaoiVS16zRe92xaxe9SsAiBwfYtm6d/zJvBiMjW8V52GDbJgh4fvRbKIRdu7aF9lEmJiFm6FBEdOqMqI8+gvSOkcJRL0GrUECW75q5168X0PrNooiPR9zX30AZEwN1aipSfl+MtO0FT/c1VlVfevMmJKvXIKp375cuUKZVKpG8dBki+/ZF/OTJBvtHkyHLKlUQcOI4Kqxbh4ATx+EyZEiZ3askP48ElpZwGjDA8LxAAKvatWFZrZrBaxmHDkFbzDoGymjDh3lPaZ9b5/6yLGvWRMqqVUjfug2Z+/dDeumyYSMLCyDfKPjTteeqtDRI797Vi0mr1SJh2nTdgwR1cgoSZ84qtZjfFmW+Bn3OnDlo0KABgv6bxjFt2jT07NkTTZs2xdSpUzFp0iRYW1vDzMxMN2rt4OCA0aNHQyKRoHPnzggICMD48ePx008/Ye3atVCpVJgxYwacnZ3x7bffYvDgwRAKhahQoQLGjh2LxMREjBs3DkuWLIFQKMQPP/wAIG99+dixYzF//ny9GNetW4dDhw5BKBRCLBZj8eLFEIvF6NSpE0aNGoWrV6+iZs2apf7ZCIVCTJ48GcOGDYNWq4WNjQ3mzp2LnJwco/ETERG9iSR/rkPuv3mVr7VyOZLmL4Bdhw4w9/EBkFflvcKa1UjfuRNCGxs4f/ppmU1VLWuOvXohbes2qFPy9uW2CAyAbbt2Rfaz8PeHz4oVkPy5Flq1Bs6ffFLkVmuJs2Yh979fmuWhYYj7fiwq/XOk1EayBebmsKheDfIHz7aEsqpdu1Su/bqlb9tmsM1a2vbtcOrb12h763r14PrN10hZ9b+8Yl/P9VVGxyBj9+68LbZeUPLixUj9X96AjOz2HcjDwlCxkAcGlEedmgpFdDQEIiHMypcvk3tknzsHkZMzbN97D/KHDyGwsoIirOARdbGrKxLnzIb05i2IHBzg8esvsGtb+MO21/HzTmBtDW0By2/MKlSAZfVqSFm6LN8LZnrF7oQ2NjCrVAny53bAsnvvPaT8bzWSFy0CNBoI7e3hu3kTLAMDoZXLDQpLKmJiSu9NvSUE2uIMK5eRzZs3o1OnTnB2dsaiRYtgZmaGb7755nWF80Z7+PAhqhl5Ikclw8+RXjV+5+hVex3fuZgRXyHn1Cm9cz6r/nhzR8mLoJJIkL5tGwTWNnDq0xvCl91nuAAhLVpAkyrROxd48QLEpbgOVR4aiviJP0B2/z6sGzaE55zZMPPyKtE1TPHnXOr6DUiapT9yZ92oEXw3bjDaXiOXI6pPX6NbywGAy4jhcPvOcIpycYV37GSwpCHg7BmYPd0DmwzkXL6MmC+G6RJGp8GDUH7SJACl951LXrwYKctX6J8UiQpcf26USIQq169BaGkJVXIyIBYbLIlQZ+cgql+/Z4m/UGiwpOJVqrBxA1KWLEHulav6LwgEBg+2YGYGaLUQOTvD8aOecB78CUKbN9drZx4YgEoH8irexwwbhpyz53SvOfbrC4+3YHer4nznivu9fK2LhVxcXDB06FD0798fwcHBGGBkOggRERG92fLviQsAQse3c82uRqHAk0mTkbx4CZLmzkVSvll7RZHeuoXcGzeLtSwPSpXBKWP7M78MrVqtK+SkUSigzZeYqLOzkXnkH+TevFmq9y1rTr17Qfj8emWBAOW+/77A9lnHjxeYnAusrODQvftLxWOer0CdyMEBIkfHl7rm2y5p0SK90dy0jZugkkgK6VEyWpUKqevWG75QkuT8v/aZR/5BzGefI7RVa4Q2b4G4ceP1mmhyc6B+PvaXTc6fr18lFsN9yhQISvCgUBEZCXW6kSLVxn4uKZWASgV1UhKs69WDPCTYoJ0y9rHuz15z58Kpf39Y1akDl2HD4M6ZwgZeyTZrBenYsSM6duz4OkMgIiKiMqYwsmYy9+pVWNeu9RqiKVsZu/cg+/TpvAONBmlbtsLug46wadK40H5ahQIxw75E7uW8KetW9eqhwp9rISygcvLTPvlpcnIgtLB44fjzi58wEfLQUACA7M4dPJk6Fb5//gkAkEdEIHrAQKj/W3/t0LMnPGfOKLV7lyWhtTX8/z4AyYaN0GRlwrF3H1jVKng5oybHcCqwZe3asK5XD469e8GiYsWXisexX19knz+vS/4cenxoUPma9BkUVtRqoU5PL70ZJEXVkxIKIXZxQbmx3yN50W9QJSQU2FQWHIycCxd0cWYeOAC79u1g/18elLl/v36C/iLMzGD3/vuARoOsI0eenVepkHPxQoHT2Y2xbtQY6bv3lDgE6e07cP7/9u4+uKZ73+P4Z0t2HnYSIZIgXFqUaqKN0NZomw7RkZYZD4kQEepMPZw7rnNpU64qnUpVmnGoYLSdqOe2Eg6lSlscuUcNQ6ItIyijFedICCIS8rjuHzndV5o0kQfJ2rxff9lr79/ev7Xnm2V91u+3f2vixCoj7da7bn3n1KqV2s17u87v/TBxrOU2AQCA46lu1KW8jqNQDqK6ixG1rcYuSTe//dYezqWKBchu7txZQ4uK34hX3Vj7IrX3yigurjJqfOfE/y+am5ucbA/nkpS3dauKzp9vtM+/36xt26pt3Btq/+67NYZzSbI4V72jju2ZZ9T2f2bL9d9362mIG19srjQym7f9S5VzC6oaVbdiu8VqbbT3tzg5qc3EidU+Z+vXT48fz1C3tANqNWyYOq9bq1ajIuX54ovynT69Uj88w8J0567FsH9zc883d33YPUayGv6+/V9/XR3/ulito0ZVec64U/VWljUpOntGnqEv1KmNJNme7isnTw/5zZxpX/izRcuW6rB0SZ3f62FGQAcAAPeVR79+Vbc9X/eTP0fgFVZ5QTiLq+s9nej+tqjc3Uqr2XY376FDKz1269WrUW/3ZXFxkXvv3pW22e6aCVDdTxeq2/ZAqCZANeZMhd9fxCm7cUNlN6q/lRYqtI4eU+mxW1CgfeHJxuL3X9PUaXWy/P4yXd6jIuXep49ax8Sow5K/yuLiYr9rk0unTmq/YIH+46NV8vvPP+uxQ98r4MOlFbdUW7FcbtUsNm3rE2L/t/fwYVXu616dFl5e6rR2rdx+v1ijs7NaRUZUvG/fvhX3Sr9L65gYef/7eUlq4ekpOTnJ4uKiFtX8lMIoKpLva6+pdWxslQuBzv5+cu3ZUy6PPiqvwYPl3K6dnP395T97lv1Y7zvpNT128B96JGWzun9/UG6NcBHrYdKsU9wBAMCDz//NOBUeO2afAuo9cqTcez7ezL26P2x9+6rD0iW6vmGjLG5uajN5kqzt29fazuull5Tz4TL7NFSLq6u8Bg+usY3/m3GyWK26dfAfcuvxuPzj3miUfbhbQGKiLr/zjm7/9JM8nnla7e66D3qryEjlf7fXPkPC9fHHqwaHB4TXoDBdWdZepf/6l6SKoOQ9Ynijvb9n2EBdS15tf+zWqxcLxNXC509/ksXNTbf27pNLly7y/fPU+/I5Hv37y6N//zq1cfL0lPddf79+f5mu/L17VfrPf0qSXHv0qHTHAOc2bfTotr8pf/duleXlybl9e5Xn39L1zz9T8c/n7K9rM2mSPJ59Ro9s3KDshe8rb8eXcvb1U7v58+Tk6Smp4sJa5/XrlLv6U5VevSrvYcPkNWCAvAYMkE/seJVdvyZbnz4VI9wWi0p+/VXnh4+wj7I7B7SX54CBsri4qN1bc+Q/47+V/f4i3fr73+X6WDe1nTPnnmaNOLdu3agXDB8mzbqKOxqPGVdIdUR8j2hq1ByaWnPVnFFSosJj6XL2bdMoU4IfRHdOndK1DRuksnK1jompddq1GRQcPqKbX30l53Zt5TN2bLULmz0ox7nSq1d1Y8tWGUVF8h4xvFFHa43iYl1ZuVIFaf8r18cek9/MGbLew4gqqmfGmjPKylR4+HDFzJQ+feyj7zUpLy7Wjc8/153Tp+X5/PNq+fLL96Vvxb/8ohvbtqmFm7taRUbIuU2b+/I5D7LGXMWdgP6AMOOByBHxPaKpUXNoatQcmho1h6ZGzaGpPTC3WQMAAAAAABUI6AAAAAAAmAABHQAAAAAAEyCgAwAAAABgAgR0AAAAAABMgIAOAAAAAIAJENABAAAAADABAjoAAAAAACZAQAcAAAAAwAQI6AAAAAAAmAABHQAAAAAAEyCgAwAAAABgAgR0AAAAAABMgIAOAAAAAIAJENABAAAAADABAjoAAAAAACZAQAcAAAAAwAQI6AAAAAAAmAABHQAAAAAAEyCgAwAAAABgAgR0AAAAAABMgIAOAAAAAIAJENABAAAAADABAjoAAAAAACZAQAcAAAAAwAQI6AAAAAAAmAABHQAAAAAAEyCgAwAAAABgAgR0AAAAAABMgIAOAAAAAIAJENABAAAAADABAjoAAAAAACZAQAcAAAAAwAQshmEYzd0JNNzx48fl6ura3N0AAAAAAPxOUVGRgoODa30dAR0AAAAAABNgijsAAAAAACZAQAcAAAAAwAQI6AAAAAAAmAABHQAAAAAAEyCgAwAAAABgAgR0AAAAAABMwLm5OwA0hfLycr3zzjs6ffq0XFxcFB8fr86dO1d5zeTJkxUWFqbo6GgZhqHQ0FA98sgjkqTg4GC9/vrrzdB7OKra6i4+Pl7p6eny8PCQJK1cuVIlJSV64403dOfOHfn7++v999+Xu7t7c+0CHEx9aq6srEyDBw9W9+7dJUmDBg3ShAkTmqX/cDy11dyBAwe0YsUKSdITTzyh+fPnq6ioSHFxccrNzZWHh4cSEhLk4+PTXLsAB1OfmpPEOR3qraaaO3XqlBYuXGh/7fHjx7VixQoFBQXV/3zOAB4Ce/bsMWbNmmUYhmFkZGQYU6dOrfKaxYsXG5GRkcamTZsMwzCMCxcuGFOmTGnSfuLBUlvdjRkzxsjNza20bcGCBcaWLVsMwzCMjz76yPj000+bpK94MNSn5g4ePGi8++67TdZHPFhqqrn8/HxjyJAh9pr7+OOPjdzcXGP16tXGsmXLDMMwjJ07dxoLFixo+o7DYdWn5jinQ0PcS44wDMPYtWuXMXPmTMMwGnY+xxR3PBSOHTumF154QVLFVdMTJ05Uen737t2yWCwKDQ21bzt58qSys7MVGxurSZMm6fz5803aZzi+muquvLxcv/zyi+bNm6cxY8YoNTW1SpvQ0FB9//33Td9xOKz61NyJEyd08uRJjRs3TtOnT1dOTk6z9B2Oqaaay8jIUPfu3ZWQkKCxY8fK19dXPj4+VY5zhw4dapa+wzHVp+Y4p0ND1JYjJKmwsFBJSUl66623qrSp6/kcU9zxULh165Y8PT3tj52cnFRaWipnZ2edOXNGO3fu1LJly+xToiTJz89PkydP1ssvv6yjR48qLi5OW7ZsaY7uw0HVVHeFhYUaN26cJk6cqLKyMo0fP15BQUG6deuWvLy8JEkeHh7Kz89vru7DAdWn5rp06aKgoCD1799fX375peLj47Vs2bJm3As4kppq7vr16zp8+LC2bdsmm82mmJgYBQcHc5xDg9Sn5jinQ0PUVHO/SU1NVXh4uP3nOg05zhHQ8VDw9PRUQUGB/XF5ebn9j2rbtm3Kzs7WhAkTdOnSJVmtVnXo0EFPP/20nJycJEl9+/ZVdna2DMOQxWJpln2A46mp7tzd3TV+/Hj775H69eunzMxMexs3NzcVFBSoZcuWzdJ3OKb61NygQYPs21566SXCOeqkpppr1aqVevXqJT8/P0kV/5eeOnWqUhuOc6ir+tTcgAEDOKdDvdVUc7/ZsWNHpf8/G3I+xxR3PBRCQkKUlpYmqWLxht8WQ5KkN998UykpKVq/fr1GjBihV199VaGhoVq+fLnWrl0rScrMzFRAQAAHctRJTXV34cIFjR07VmVlZSopKVF6eroCAwMVEhKiAwcOSJLS0tLUp0+fZuk7HFN9am7u3Lnas2ePJOnQoUMKDAxslr7DMdVUc0FBQTpz5oyuXbum0tJS/fDDD+rWrRvHOTRIfWqOczo0RE01J0n5+fkqLi5W+/btK7Wp73HOYhiG0Qj9Bkztt9UXz5w5I8MwtHDhQqWlpalTp04KCwuzvy4pKUm+vr6Kjo5WXl6e4uLiVFhYKCcnJ82bN09du3Ztxr2Ao6mt7j755BPt3r1bVqtVw4YNU3R0tK5evapZs2apoKBArVu31uLFi2Wz2Zp7V+Ag6lNzFy9e1Jw5cyRVjLLHx8fL39+/mfcEjqK2mvvqq6+UnJwsSQoPD9fkyZN1+/ZtzZo1S1euXJHVatXixYvtI55AbepTc5zToSFqq7kff/xRq1at0sqVK+1tGnI+R0AHAAAAAMAEmOIOAAAAAIAJENABAAAAADABAjoAAAAAACZAQAcAAAAAwAQI6AAAAAAAmAABHQAA2MXExOjQoUOVtsXHxyslJaXa12dlZSkqKkqSNGPGDBUXF1d6Pi0tTbNnz/7DzysqKrK/99atW7V3796GdF+StGvXLgUHBys7O9u+LSkpSZ999lml10VFRSkrK0uSdPToUU2cOFGxsbGKiIjQxo0bG9wPAADqioAOAADsoqKitH37dvvj4uJi7d+/X0OGDKm17ZIlS+Ti4lKnz7ty5Yo9oI8cOVJhYWF163A1UlJSNG7cOG3evPmeXn/x4kXFx8crMTFR69ev18aNG7V9+3alpaU1uC8AANSFc3N3AAAAmEd4eLiWLl2q27dvy93dXXv37tVzzz0nm82mI0eOaPny5ZKkO3fuKCEhQVar1d524MCB+vrrr5WVlaU5c+bI3d1d7u7u8vb2liRt2LBB33zzjUpLS+Xl5aWkpCStWrVKP//8s5YvXy7DMOTr66vo6GgtWrRIx44dkyQNHTpUEyZM0OzZs+Xi4qJLly4pJydHixYtUmBgYKX+X7x4UXl5eZoyZYpGjBihqVOnVupjdbZv367hw4fL19dXkuTm5qbk5GTZbLZG+14BALgXjKADAAA7V1dXhYWF6dtvv5VUMe189OjRkqSzZ88qMTFR69at08CBA7V79+5q3+PDDz/U9OnTtWbNGvXu3VuSVF5erhs3bmjNmjXatGmTSktL9dNPP2nq1Knq1q2bpk2bZm+/f/9+ZWVlafPmzdq0aZN27typ06dPS5ICAgKUnJys2NhYffHFF1U+OzU1VREREfLy8lJwcLB9P/6IxWJRTk6OOnbsWGm7l5eXnJyc7vFbAwCgcTCCDgAAKhk1apQ++OADPfvss7p586Z9lLpt27Z67733ZLPZlJ2drZCQkGrbnz17Vk8++aQkKSQkROfPn1eLFi1ktVo1c+ZM2Ww2Xb58WaWlpdW2P3funPr27SuLxSKr1aqnnnpK586dkyT17NlTktSuXTulp6dXaldWVqYdO3aoQ4cO2rdvn/Ly8rRhwwa98sorcnV1rfL7+MLCQrm5uSkgIECXL1+u9FxmZqYMw7B/HgAATYERdAAAUEmPHj1UUFCgdevWKSIiwr597ty5WrhwoRYtWiR/f38ZhlFt+y5duigjI0OSdOLECUkVgfe7777T0qVL9fbbb6u8vFyGYahFixYqLy+v1L5r16726e0lJSXKyMhQ586dJVWMeP+RAwcOKCgoSOvXr1dycrJSU1OVm5urzMxMBQYGat++ffaLAr/++quKi4vVpk0bDR06VCkpKbp27ZokqaCgQPPmzVNOTk59vj4AAOqNEXQAAFBFRESEEhMTtX//fvu2YcOGKSoqSi1btpSvr+8fBtj58+drxowZSk5Olo+Pj1xdXdW5c2e5u7tr5MiRcnFxkZ+fn3JyctS7d2+VlJQoMTFRbm5ukqQBAwboyJEjGj16tEpKShQeHl7lt+bV2bx5s0aNGlVpW2RkpDZu3KgFCxYoPT1dI0eOlKenpwzDUEJCgiSpY8eOiouL07Rp0+Tk5KSCggJFRkbqxRdfrO/XBwBAvViMP7r8DQAAAAAAmgxT3AEAAAAAMAECOgAAAAAAJkBABwAAAADABAjoAAAAAACYAAEdAAAAAAATIKADAAAAAGACBHQAAAAAAEzg/wCoNNsZfBmKhgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1008x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(14,6))\n",
    "ax = sns.stripplot(x='Validation AUC', y='Model', data=df)\n",
    "ax.set_title('Mean Cross-Validation Performance (AUC)', fontsize=14)\n",
    "ax.text(x=.02, y=.85, s='\\n'.join([f'{m:<35}' for m, s in top_scores.items()]),\n",
    "            transform=ax.transAxes, fontsize=12, \n",
    "            bbox=dict(facecolor='white', edgecolor='grey', boxstyle='round,pad=0.5'))\n",
    "ax.text(x=.15, y=.85, s='\\n'.join([f'{s:>8.2%}' for m, s in top_scores.items()]),\n",
    "            transform=ax.transAxes, fontsize=12)\n",
    "plt.tight_layout();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Combine Model Results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Collect results for each model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-24T17:37:12.555176Z",
     "start_time": "2018-10-24T17:37:12.519696Z"
    }
   },
   "outputs": [],
   "source": [
    "cols = ['model', 'data', 'train', 'valid', 'boosting', 'learning_rate', 'max_depth', 'num_leaves',\n",
    "        'colsample_bytree', 'min_gain_to_split', 'rounds', 'time']\n",
    "\n",
    "with pd.HDFStore('results.h5') as store:\n",
    "    xgb = (store['xgboost/dummies']\n",
    "           .rename(columns={'booster': 'boosting', 'gamma': 'min_gain_to_split'})\n",
    "           .assign(model='xgb', data='dummies'))\n",
    "\n",
    "    lgb_dummies = (store['lightgbm/dummies']\n",
    "                   .assign(model='lightgbm', data='dummies'))\n",
    "    lgb_dummies.boosting = lgb_dummies.boosting.str.replace('gbdt', 'gbtree')\n",
    "\n",
    "    lgb_factors = (store['lightgbm/factors']\n",
    "                   .assign(model='lightgbm', data='factors'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 216 entries, 0 to 215\n",
      "Data columns (total 11 columns):\n",
      "rounds               216 non-null float64\n",
      "train                216 non-null float64\n",
      "valid                216 non-null float64\n",
      "boosting             216 non-null object\n",
      "learning_rate        216 non-null float64\n",
      "min_gain_to_split    216 non-null int64\n",
      "max_depth            216 non-null int64\n",
      "colsample_bytree     216 non-null float64\n",
      "time                 216 non-null float64\n",
      "model                216 non-null object\n",
      "data                 216 non-null object\n",
      "dtypes: float64(6), int64(2), object(3)\n",
      "memory usage: 20.2+ KB\n"
     ]
    }
   ],
   "source": [
    "xgb.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 396 entries, 0 to 395\n",
      "Data columns (total 16 columns):\n",
      "boosting             396 non-null object\n",
      "colsample_bytree     396 non-null float64\n",
      "gamma                180 non-null float64\n",
      "learning_rate        396 non-null float64\n",
      "max_depth            396 non-null int64\n",
      "min_gain_to_split    396 non-null int64\n",
      "num_leaves           396 non-null int64\n",
      "rounds               396 non-null float64\n",
      "rounds_std           180 non-null float64\n",
      "time                 396 non-null float64\n",
      "train                396 non-null float64\n",
      "train_std            180 non-null float64\n",
      "valid                396 non-null float64\n",
      "valid_std            180 non-null float64\n",
      "model                396 non-null object\n",
      "data                 396 non-null object\n",
      "dtypes: float64(10), int64(3), object(3)\n",
      "memory usage: 52.6+ KB\n"
     ]
    }
   ],
   "source": [
    "lgb_dummies.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 396 entries, 0 to 395\n",
      "Data columns (total 16 columns):\n",
      "boosting             396 non-null object\n",
      "colsample_bytree     396 non-null float64\n",
      "gamma                180 non-null float64\n",
      "learning_rate        396 non-null float64\n",
      "max_depth            396 non-null int64\n",
      "min_gain_to_split    396 non-null int64\n",
      "num_leaves           396 non-null int64\n",
      "rounds               396 non-null float64\n",
      "rounds_std           180 non-null float64\n",
      "time                 396 non-null float64\n",
      "train                396 non-null float64\n",
      "train_std            180 non-null float64\n",
      "valid                396 non-null float64\n",
      "valid_std            180 non-null float64\n",
      "model                396 non-null object\n",
      "data                 396 non-null object\n",
      "dtypes: float64(10), int64(3), object(3)\n",
      "memory usage: 52.6+ KB\n"
     ]
    }
   ],
   "source": [
    "lgb_factors.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Combine LightGBM and XGBoost results that have largely the same hyperparameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 1008 entries, 0 to 1007\n",
      "Data columns (total 11 columns):\n",
      "train                1008 non-null float64\n",
      "valid                1008 non-null float64\n",
      "boosting             1008 non-null object\n",
      "learning_rate        1008 non-null float64\n",
      "max_depth            1008 non-null int64\n",
      "num_leaves           792 non-null float64\n",
      "colsample_bytree     1008 non-null float64\n",
      "min_gain_to_split    1008 non-null int64\n",
      "rounds               1008 non-null float64\n",
      "time                 1008 non-null float64\n",
      "model_type           1008 non-null object\n",
      "dtypes: float64(7), int64(2), object(2)\n",
      "memory usage: 86.7+ KB\n"
     ]
    }
   ],
   "source": [
    "lgb_xgb_combined = (lgb_dummies\n",
    "                    .append(lgb_factors)\n",
    "                    .append(xgb, ignore_index=True)).loc[:, cols]\n",
    "\n",
    "lgb_xgb_combined['model_type'] = lgb_xgb_combined.model.str.capitalize(\n",
    ").str.cat(lgb_xgb_combined.data.str.capitalize(), sep=' ')\n",
    "lgb_xgb_combined = lgb_xgb_combined.drop(['model', 'data'], axis=1)\n",
    "\n",
    "lgb_xgb_combined.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Best Performing Models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A more detailed look at the top-performing models for the LightGBM and XGBoost models shows that the LightGBM Factors model achieves nearly as good a performance as the other two models with much lower model complexity. \n",
    "\n",
    "It only consists on average of 41 trees up to three levels deep with no more than eight leaves each, while also using regularization in the form of min_gain_to_split. It overfits significantly less on the training set, with a train AUC only slightly above the validation AUC. \n",
    "\n",
    "It also trains much faster, taking only 18 seconds per model because of its lower complexity. In practice, this model would be preferable since it is more likely to produce good out-of-sample performance. The details are shown in the following table:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Metric            | Lightgbm Dummies | Xgb Dummies | Lightgbm Factors | CatBoost |\n",
    "|-------------------|------------------|-------------|------------------|----------|\n",
    "| Validation AUC    | 68.57%           | 68.36%      | 68.32%           | 68.70%   |\n",
    "| Train AUC         | 82.35%           | 79.81%      | 72.12%           | 73.61%   |\n",
    "| learning_rate     | 0.1              | 0.1         | 0.3              |          |\n",
    "| max_depth         | 13               | 9           | 3                | 12       |\n",
    "| num_leaves        | 8192             |             | 8                |          |\n",
    "| colsample_bytree  | 0.8              | 1           | 1                |          |\n",
    "| min_gain_to_split | 0                | 1           | 0                |          |\n",
    "| rounds            | 44.42            | 59.17       | 41.00            | 22.33    |\n",
    "| time              | 86.55            | 85.37       | 18.78            | 229.66   |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-24T17:37:15.433285Z",
     "start_time": "2018-10-24T17:37:15.421969Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>train</th>\n",
       "      <th>valid</th>\n",
       "      <th>boosting</th>\n",
       "      <th>learning_rate</th>\n",
       "      <th>max_depth</th>\n",
       "      <th>num_leaves</th>\n",
       "      <th>colsample_bytree</th>\n",
       "      <th>min_gain_to_split</th>\n",
       "      <th>rounds</th>\n",
       "      <th>time</th>\n",
       "      <th>model_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.823529</td>\n",
       "      <td>0.685673</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>0.1</td>\n",
       "      <td>13</td>\n",
       "      <td>8192.0</td>\n",
       "      <td>0.8</td>\n",
       "      <td>0</td>\n",
       "      <td>44.416667</td>\n",
       "      <td>86.548636</td>\n",
       "      <td>Lightgbm Dummies</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.803322</td>\n",
       "      <td>0.683760</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>0.1</td>\n",
       "      <td>12</td>\n",
       "      <td>4096.0</td>\n",
       "      <td>0.8</td>\n",
       "      <td>0</td>\n",
       "      <td>34.916667</td>\n",
       "      <td>63.578490</td>\n",
       "      <td>Lightgbm Dummies</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>792</th>\n",
       "      <td>0.798142</td>\n",
       "      <td>0.683618</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>0.1</td>\n",
       "      <td>9</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>59.166667</td>\n",
       "      <td>85.372013</td>\n",
       "      <td>Xgb Dummies</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>793</th>\n",
       "      <td>0.798142</td>\n",
       "      <td>0.683618</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>0.1</td>\n",
       "      <td>9</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>5</td>\n",
       "      <td>59.166667</td>\n",
       "      <td>94.299514</td>\n",
       "      <td>Xgb Dummies</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>794</th>\n",
       "      <td>0.798142</td>\n",
       "      <td>0.683618</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>0.1</td>\n",
       "      <td>9</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>59.166667</td>\n",
       "      <td>75.323257</td>\n",
       "      <td>Xgb Dummies</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>396</th>\n",
       "      <td>0.721158</td>\n",
       "      <td>0.683226</td>\n",
       "      <td>gbdt</td>\n",
       "      <td>0.3</td>\n",
       "      <td>3</td>\n",
       "      <td>8.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>41.000000</td>\n",
       "      <td>18.782672</td>\n",
       "      <td>Lightgbm Factors</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.801764</td>\n",
       "      <td>0.682428</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>0.1</td>\n",
       "      <td>11</td>\n",
       "      <td>2048.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>53.916667</td>\n",
       "      <td>71.732526</td>\n",
       "      <td>Lightgbm Dummies</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.785657</td>\n",
       "      <td>0.682390</td>\n",
       "      <td>dart</td>\n",
       "      <td>0.1</td>\n",
       "      <td>13</td>\n",
       "      <td>8192.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>54.166667</td>\n",
       "      <td>171.725467</td>\n",
       "      <td>Lightgbm Dummies</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.729307</td>\n",
       "      <td>0.682187</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>0.3</td>\n",
       "      <td>7</td>\n",
       "      <td>128.0</td>\n",
       "      <td>0.6</td>\n",
       "      <td>5</td>\n",
       "      <td>18.833333</td>\n",
       "      <td>15.769092</td>\n",
       "      <td>Lightgbm Dummies</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.774811</td>\n",
       "      <td>0.682066</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>0.1</td>\n",
       "      <td>9</td>\n",
       "      <td>512.0</td>\n",
       "      <td>0.8</td>\n",
       "      <td>0</td>\n",
       "      <td>53.916667</td>\n",
       "      <td>46.393234</td>\n",
       "      <td>Lightgbm Dummies</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        train     valid boosting  learning_rate  max_depth  num_leaves  \\\n",
       "0    0.823529  0.685673   gbtree            0.1         13      8192.0   \n",
       "1    0.803322  0.683760   gbtree            0.1         12      4096.0   \n",
       "792  0.798142  0.683618   gbtree            0.1          9         NaN   \n",
       "793  0.798142  0.683618   gbtree            0.1          9         NaN   \n",
       "794  0.798142  0.683618   gbtree            0.1          9         NaN   \n",
       "396  0.721158  0.683226     gbdt            0.3          3         8.0   \n",
       "2    0.801764  0.682428   gbtree            0.1         11      2048.0   \n",
       "3    0.785657  0.682390     dart            0.1         13      8192.0   \n",
       "4    0.729307  0.682187   gbtree            0.3          7       128.0   \n",
       "5    0.774811  0.682066   gbtree            0.1          9       512.0   \n",
       "\n",
       "     colsample_bytree  min_gain_to_split     rounds        time  \\\n",
       "0                 0.8                  0  44.416667   86.548636   \n",
       "1                 0.8                  0  34.916667   63.578490   \n",
       "792               1.0                  1  59.166667   85.372013   \n",
       "793               1.0                  5  59.166667   94.299514   \n",
       "794               1.0                  0  59.166667   75.323257   \n",
       "396               1.0                  0  41.000000   18.782672   \n",
       "2                 1.0                  0  53.916667   71.732526   \n",
       "3                 1.0                  0  54.166667  171.725467   \n",
       "4                 0.6                  5  18.833333   15.769092   \n",
       "5                 0.8                  0  53.916667   46.393234   \n",
       "\n",
       "           model_type  \n",
       "0    Lightgbm Dummies  \n",
       "1    Lightgbm Dummies  \n",
       "792       Xgb Dummies  \n",
       "793       Xgb Dummies  \n",
       "794       Xgb Dummies  \n",
       "396  Lightgbm Factors  \n",
       "2    Lightgbm Dummies  \n",
       "3    Lightgbm Dummies  \n",
       "4    Lightgbm Dummies  \n",
       "5    Lightgbm Dummies  "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lgb_xgb_combined.sort_values('valid', ascending=False).head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-24T17:37:17.634444Z",
     "start_time": "2018-10-24T17:37:17.624640Z"
    }
   },
   "outputs": [],
   "source": [
    "lgb_xgb_combined.groupby('model_type').apply(lambda x: x.sort_values('valid', ascending=False).head(1)).reset_index(-1, drop=True).T.to_csv('best_models.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Decision Tree for all models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-24T17:37:18.798410Z",
     "start_time": "2018-10-24T17:37:18.795368Z"
    }
   },
   "outputs": [],
   "source": [
    "reg_tree = DecisionTreeRegressor(criterion='mse',\n",
    "                                 splitter='best',\n",
    "                                 min_samples_split=2,\n",
    "                                 min_samples_leaf=2,\n",
    "                                 min_weight_fraction_leaf=0.0,\n",
    "                                 max_features=None,\n",
    "                                 random_state=42,\n",
    "                                 max_leaf_nodes=None,\n",
    "                                 min_impurity_decrease=0.0,\n",
    "                                 min_impurity_split=None,\n",
    "                                 presort=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-24T17:37:32.641061Z",
     "start_time": "2018-10-24T17:37:32.622695Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeRegressor(criterion='mse', max_depth=None, max_features=None,\n",
       "           max_leaf_nodes=None, min_impurity_decrease=0.0,\n",
       "           min_impurity_split=None, min_samples_leaf=2,\n",
       "           min_samples_split=2, min_weight_fraction_leaf=0.0,\n",
       "           presort=False, random_state=42, splitter='best')"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_score = lgb_xgb_combined.valid\n",
    "feature_data = lgb_xgb_combined.drop(['train', 'valid', 'time', 'rounds'], axis=1).fillna(-1)\n",
    "feature_data = pd.get_dummies(feature_data, columns=feature_data.columns)\n",
    "features = feature_data.columns\n",
    "reg_tree.fit(X=feature_data, y=test_score)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-24T17:37:35.545121Z",
     "start_time": "2018-10-24T17:37:35.512023Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.40.1 (20161225.0304)\n",
       " -->\n",
       "<!-- Title: Tree Pages: 1 -->\n",
       "<svg width=\"2978pt\" height=\"564pt\"\n",
       " viewBox=\"0.00 0.00 2977.50 564.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 560)\">\n",
       "<title>Tree</title>\n",
       "<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-4,4 -4,-560 2973.5,-560 2973.5,4 -4,4\"/>\n",
       "<!-- 0 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>0</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.831373\" stroke=\"#000000\" d=\"M1673.5,-556C1673.5,-556 1533.5,-556 1533.5,-556 1527.5,-556 1521.5,-550 1521.5,-544 1521.5,-544 1521.5,-500 1521.5,-500 1521.5,-494 1527.5,-488 1533.5,-488 1533.5,-488 1673.5,-488 1673.5,-488 1679.5,-488 1685.5,-494 1685.5,-500 1685.5,-500 1685.5,-544 1685.5,-544 1685.5,-550 1679.5,-556 1673.5,-556\"/>\n",
       "<text text-anchor=\"start\" x=\"1529.5\" y=\"-540.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">learning_rate_0.01 ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"1565.5\" y=\"-525.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.001</text>\n",
       "<text text-anchor=\"start\" x=\"1555\" y=\"-510.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 1008</text>\n",
       "<text text-anchor=\"start\" x=\"1562.5\" y=\"-495.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.656</text>\n",
       "</g>\n",
       "<!-- 1 -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>1</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.917647\" stroke=\"#000000\" d=\"M1254.5,-452C1254.5,-452 1130.5,-452 1130.5,-452 1124.5,-452 1118.5,-446 1118.5,-440 1118.5,-440 1118.5,-396 1118.5,-396 1118.5,-390 1124.5,-384 1130.5,-384 1130.5,-384 1254.5,-384 1254.5,-384 1260.5,-384 1266.5,-390 1266.5,-396 1266.5,-396 1266.5,-440 1266.5,-440 1266.5,-446 1260.5,-452 1254.5,-452\"/>\n",
       "<text text-anchor=\"start\" x=\"1126.5\" y=\"-436.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">boosting_gbtree ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"1162\" y=\"-421.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"1147.5\" y=\"-406.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 672</text>\n",
       "<text text-anchor=\"start\" x=\"1155\" y=\"-391.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.67</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;1 -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>0&#45;&gt;1</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1521.3158,-501.204C1450.28,-483.229 1348.0481,-457.3601 1276.4899,-439.2529\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1277.1558,-435.8112 1266.6027,-436.7511 1275.4386,-442.5973 1277.1558,-435.8112\"/>\n",
       "<text text-anchor=\"middle\" x=\"1279.4038\" y=\"-454.5251\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">True</text>\n",
       "</g>\n",
       "<!-- 506 -->\n",
       "<g id=\"node33\" class=\"node\">\n",
       "<title>506</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.654902\" stroke=\"#000000\" d=\"M2071.5,-452C2071.5,-452 1911.5,-452 1911.5,-452 1905.5,-452 1899.5,-446 1899.5,-440 1899.5,-440 1899.5,-396 1899.5,-396 1899.5,-390 1905.5,-384 1911.5,-384 1911.5,-384 2071.5,-384 2071.5,-384 2077.5,-384 2083.5,-390 2083.5,-396 2083.5,-396 2083.5,-440 2083.5,-440 2083.5,-446 2077.5,-452 2071.5,-452\"/>\n",
       "<text text-anchor=\"start\" x=\"1907.5\" y=\"-436.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">colsample_bytree_1.0 ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"1953.5\" y=\"-421.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.001</text>\n",
       "<text text-anchor=\"start\" x=\"1946.5\" y=\"-406.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 336</text>\n",
       "<text text-anchor=\"start\" x=\"1950.5\" y=\"-391.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.628</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;506 -->\n",
       "<g id=\"edge32\" class=\"edge\">\n",
       "<title>0&#45;&gt;506</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1685.7848,-499.9443C1745.2243,-484.012 1825.8884,-462.3907 1889.4344,-445.3578\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1890.7274,-448.6348 1899.4803,-442.6651 1888.915,-441.8735 1890.7274,-448.6348\"/>\n",
       "<text text-anchor=\"middle\" x=\"1886.9821\" y=\"-460.6168\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">False</text>\n",
       "</g>\n",
       "<!-- 2 -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>2</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.909804\" stroke=\"#000000\" d=\"M722.5,-348C722.5,-348 562.5,-348 562.5,-348 556.5,-348 550.5,-342 550.5,-336 550.5,-336 550.5,-292 550.5,-292 550.5,-286 556.5,-280 562.5,-280 562.5,-280 722.5,-280 722.5,-280 728.5,-280 734.5,-286 734.5,-292 734.5,-292 734.5,-336 734.5,-336 734.5,-342 728.5,-348 722.5,-348\"/>\n",
       "<text text-anchor=\"start\" x=\"558.5\" y=\"-332.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">colsample_bytree_1.0 ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"612\" y=\"-317.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"597.5\" y=\"-302.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 468</text>\n",
       "<text text-anchor=\"start\" x=\"601.5\" y=\"-287.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.669</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;2 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>1&#45;&gt;2</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1118.3246,-403.9741C1021.9336,-385.7474 853.1637,-353.8346 744.6957,-333.3243\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"745.1936,-329.8565 734.7174,-331.4375 743.893,-336.7346 745.1936,-329.8565\"/>\n",
       "</g>\n",
       "<!-- 367 -->\n",
       "<g id=\"node18\" class=\"node\">\n",
       "<title>367</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.929412\" stroke=\"#000000\" d=\"M1247.5,-348C1247.5,-348 1137.5,-348 1137.5,-348 1131.5,-348 1125.5,-342 1125.5,-336 1125.5,-336 1125.5,-292 1125.5,-292 1125.5,-286 1131.5,-280 1137.5,-280 1137.5,-280 1247.5,-280 1247.5,-280 1253.5,-280 1259.5,-286 1259.5,-292 1259.5,-292 1259.5,-336 1259.5,-336 1259.5,-342 1253.5,-348 1247.5,-348\"/>\n",
       "<text text-anchor=\"start\" x=\"1133.5\" y=\"-332.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">max_depth_3 ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"1162\" y=\"-317.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"1147.5\" y=\"-302.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 204</text>\n",
       "<text text-anchor=\"start\" x=\"1151.5\" y=\"-287.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.673</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;367 -->\n",
       "<g id=\"edge17\" class=\"edge\">\n",
       "<title>1&#45;&gt;367</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1192.5,-383.9465C1192.5,-375.776 1192.5,-366.9318 1192.5,-358.3697\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1196.0001,-358.13 1192.5,-348.13 1189.0001,-358.13 1196.0001,-358.13\"/>\n",
       "</g>\n",
       "<!-- 3 -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>3</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.917647\" stroke=\"#000000\" d=\"M405.5,-244C405.5,-244 295.5,-244 295.5,-244 289.5,-244 283.5,-238 283.5,-232 283.5,-232 283.5,-188 283.5,-188 283.5,-182 289.5,-176 295.5,-176 295.5,-176 405.5,-176 405.5,-176 411.5,-176 417.5,-182 417.5,-188 417.5,-188 417.5,-232 417.5,-232 417.5,-238 411.5,-244 405.5,-244\"/>\n",
       "<text text-anchor=\"start\" x=\"291.5\" y=\"-228.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">max_depth_3 ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"320\" y=\"-213.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"305.5\" y=\"-198.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 270</text>\n",
       "<text text-anchor=\"start\" x=\"309.5\" y=\"-183.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.671</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;3 -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>2&#45;&gt;3</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M550.1094,-281.0938C510.6913,-267.0544 465.2189,-250.8588 427.6407,-237.4748\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"428.3603,-234.0157 417.7656,-233.9576 426.0116,-240.61 428.3603,-234.0157\"/>\n",
       "</g>\n",
       "<!-- 226 -->\n",
       "<g id=\"node11\" class=\"node\">\n",
       "<title>226</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.901961\" stroke=\"#000000\" d=\"M747.5,-244C747.5,-244 537.5,-244 537.5,-244 531.5,-244 525.5,-238 525.5,-232 525.5,-232 525.5,-188 525.5,-188 525.5,-182 531.5,-176 537.5,-176 537.5,-176 747.5,-176 747.5,-176 753.5,-176 759.5,-182 759.5,-188 759.5,-188 759.5,-232 759.5,-232 759.5,-238 753.5,-244 747.5,-244\"/>\n",
       "<text text-anchor=\"start\" x=\"533.5\" y=\"-228.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">model_type_Lightgbm Factors ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"612\" y=\"-213.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"597.5\" y=\"-198.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 198</text>\n",
       "<text text-anchor=\"start\" x=\"601.5\" y=\"-183.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.668</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;226 -->\n",
       "<g id=\"edge10\" class=\"edge\">\n",
       "<title>2&#45;&gt;226</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M642.5,-279.9465C642.5,-271.776 642.5,-262.9318 642.5,-254.3697\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"646.0001,-254.13 642.5,-244.13 639.0001,-254.13 646.0001,-254.13\"/>\n",
       "</g>\n",
       "<!-- 4 -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>4</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.921569\" stroke=\"#000000\" d=\"M203,-140C203,-140 12,-140 12,-140 6,-140 0,-134 0,-128 0,-128 0,-84 0,-84 0,-78 6,-72 12,-72 12,-72 203,-72 203,-72 209,-72 215,-78 215,-84 215,-84 215,-128 215,-128 215,-134 209,-140 203,-140\"/>\n",
       "<text text-anchor=\"start\" x=\"8\" y=\"-124.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">model_type_Xgb Dummies ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"77\" y=\"-109.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"62.5\" y=\"-94.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 240</text>\n",
       "<text text-anchor=\"start\" x=\"66.5\" y=\"-79.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.671</text>\n",
       "</g>\n",
       "<!-- 3&#45;&gt;4 -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>3&#45;&gt;4</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M283.4559,-181.3062C256.5822,-169.8047 225.2271,-156.3853 196.4908,-144.0866\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"197.8324,-140.8537 187.2618,-140.1367 195.0781,-147.2891 197.8324,-140.8537\"/>\n",
       "</g>\n",
       "<!-- 199 -->\n",
       "<g id=\"node8\" class=\"node\">\n",
       "<title>199</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.894118\" stroke=\"#000000\" d=\"M455.5,-140C455.5,-140 245.5,-140 245.5,-140 239.5,-140 233.5,-134 233.5,-128 233.5,-128 233.5,-84 233.5,-84 233.5,-78 239.5,-72 245.5,-72 245.5,-72 455.5,-72 455.5,-72 461.5,-72 467.5,-78 467.5,-84 467.5,-84 467.5,-128 467.5,-128 467.5,-134 461.5,-140 455.5,-140\"/>\n",
       "<text text-anchor=\"start\" x=\"241.5\" y=\"-124.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">model_type_Lightgbm Factors ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"320\" y=\"-109.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"309.5\" y=\"-94.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 30</text>\n",
       "<text text-anchor=\"start\" x=\"309.5\" y=\"-79.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.667</text>\n",
       "</g>\n",
       "<!-- 3&#45;&gt;199 -->\n",
       "<g id=\"edge7\" class=\"edge\">\n",
       "<title>3&#45;&gt;199</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M350.5,-175.9465C350.5,-167.776 350.5,-158.9318 350.5,-150.3697\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"354.0001,-150.13 350.5,-140.13 347.0001,-150.13 354.0001,-150.13\"/>\n",
       "</g>\n",
       "<!-- 5 -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>5</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M86.5,-36C86.5,-36 56.5,-36 56.5,-36 50.5,-36 44.5,-30 44.5,-24 44.5,-24 44.5,-12 44.5,-12 44.5,-6 50.5,0 56.5,0 56.5,0 86.5,0 86.5,0 92.5,0 98.5,-6 98.5,-12 98.5,-12 98.5,-24 98.5,-24 98.5,-30 92.5,-36 86.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"71.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 4&#45;&gt;5 -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>4&#45;&gt;5</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M93.5815,-71.9769C89.9977,-63.2167 86.1862,-53.8995 82.7564,-45.5157\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"85.968,-44.1222 78.9422,-36.192 79.4891,-46.7727 85.968,-44.1222\"/>\n",
       "</g>\n",
       "<!-- 184 -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>184</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M158.5,-36C158.5,-36 128.5,-36 128.5,-36 122.5,-36 116.5,-30 116.5,-24 116.5,-24 116.5,-12 116.5,-12 116.5,-6 122.5,0 128.5,0 128.5,0 158.5,0 158.5,0 164.5,0 170.5,-6 170.5,-12 170.5,-12 170.5,-24 170.5,-24 170.5,-30 164.5,-36 158.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"143.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 4&#45;&gt;184 -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>4&#45;&gt;184</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M121.4185,-71.9769C125.0023,-63.2167 128.8138,-53.8995 132.2436,-45.5157\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"135.5109,-46.7727 136.0578,-36.192 129.032,-44.1222 135.5109,-46.7727\"/>\n",
       "</g>\n",
       "<!-- 200 -->\n",
       "<g id=\"node9\" class=\"node\">\n",
       "<title>200</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M346.5,-36C346.5,-36 316.5,-36 316.5,-36 310.5,-36 304.5,-30 304.5,-24 304.5,-24 304.5,-12 304.5,-12 304.5,-6 310.5,0 316.5,0 316.5,0 346.5,0 346.5,0 352.5,0 358.5,-6 358.5,-12 358.5,-12 358.5,-24 358.5,-24 358.5,-30 352.5,-36 346.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"331.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 199&#45;&gt;200 -->\n",
       "<g id=\"edge8\" class=\"edge\">\n",
       "<title>199&#45;&gt;200</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M343.1541,-71.9769C341.3038,-63.4071 339.3384,-54.3043 337.5592,-46.0638\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"340.9595,-45.228 335.4278,-36.192 334.1172,-46.7054 340.9595,-45.228\"/>\n",
       "</g>\n",
       "<!-- 211 -->\n",
       "<g id=\"node10\" class=\"node\">\n",
       "<title>211</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M418.5,-36C418.5,-36 388.5,-36 388.5,-36 382.5,-36 376.5,-30 376.5,-24 376.5,-24 376.5,-12 376.5,-12 376.5,-6 382.5,0 388.5,0 388.5,0 418.5,0 418.5,0 424.5,0 430.5,-6 430.5,-12 430.5,-12 430.5,-24 430.5,-24 430.5,-30 424.5,-36 418.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"403.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 199&#45;&gt;211 -->\n",
       "<g id=\"edge9\" class=\"edge\">\n",
       "<title>199&#45;&gt;211</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M370.9912,-71.9769C376.3819,-63.0262 382.1228,-53.4941 387.2565,-44.9703\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"390.3824,-46.5641 392.5435,-36.192 384.386,-42.9525 390.3824,-46.5641\"/>\n",
       "</g>\n",
       "<!-- 227 -->\n",
       "<g id=\"node12\" class=\"node\">\n",
       "<title>227</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.921569\" stroke=\"#000000\" d=\"M625,-140C625,-140 498,-140 498,-140 492,-140 486,-134 486,-128 486,-128 486,-84 486,-84 486,-78 492,-72 498,-72 498,-72 625,-72 625,-72 631,-72 637,-78 637,-84 637,-84 637,-128 637,-128 637,-134 631,-140 625,-140\"/>\n",
       "<text text-anchor=\"start\" x=\"494\" y=\"-124.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">num_leaves_8.0 ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"531\" y=\"-109.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"520.5\" y=\"-94.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 90</text>\n",
       "<text text-anchor=\"start\" x=\"520.5\" y=\"-79.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.671</text>\n",
       "</g>\n",
       "<!-- 226&#45;&gt;227 -->\n",
       "<g id=\"edge11\" class=\"edge\">\n",
       "<title>226&#45;&gt;227</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M615.9776,-175.9465C609.0546,-167.0578 601.5106,-157.3716 594.304,-148.1188\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"596.988,-145.8688 588.082,-140.13 591.4654,-150.1701 596.988,-145.8688\"/>\n",
       "</g>\n",
       "<!-- 292 -->\n",
       "<g id=\"node15\" class=\"node\">\n",
       "<title>292</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.882353\" stroke=\"#000000\" d=\"M779.5,-140C779.5,-140 667.5,-140 667.5,-140 661.5,-140 655.5,-134 655.5,-128 655.5,-128 655.5,-84 655.5,-84 655.5,-78 661.5,-72 667.5,-72 667.5,-72 779.5,-72 779.5,-72 785.5,-72 791.5,-78 791.5,-84 791.5,-84 791.5,-128 791.5,-128 791.5,-134 785.5,-140 779.5,-140\"/>\n",
       "<text text-anchor=\"start\" x=\"663.5\" y=\"-124.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">boosting_gbdt ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"693\" y=\"-109.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"678.5\" y=\"-94.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 108</text>\n",
       "<text text-anchor=\"start\" x=\"682.5\" y=\"-79.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.665</text>\n",
       "</g>\n",
       "<!-- 226&#45;&gt;292 -->\n",
       "<g id=\"edge14\" class=\"edge\">\n",
       "<title>226&#45;&gt;292</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M669.0224,-175.9465C675.9454,-167.0578 683.4894,-157.3716 690.696,-148.1188\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"693.5346,-150.1701 696.918,-140.13 688.012,-145.8688 693.5346,-150.1701\"/>\n",
       "</g>\n",
       "<!-- 228 -->\n",
       "<g id=\"node13\" class=\"node\">\n",
       "<title>228</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M523.5,-36C523.5,-36 493.5,-36 493.5,-36 487.5,-36 481.5,-30 481.5,-24 481.5,-24 481.5,-12 481.5,-12 481.5,-6 487.5,0 493.5,0 493.5,0 523.5,0 523.5,0 529.5,0 535.5,-6 535.5,-12 535.5,-12 535.5,-24 535.5,-24 535.5,-30 529.5,-36 523.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"508.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 227&#45;&gt;228 -->\n",
       "<g id=\"edge12\" class=\"edge\">\n",
       "<title>227&#45;&gt;228</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M541.0088,-71.9769C535.6181,-63.0262 529.8772,-53.4941 524.7435,-44.9703\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"527.614,-42.9525 519.4565,-36.192 521.6176,-46.5641 527.614,-42.9525\"/>\n",
       "</g>\n",
       "<!-- 289 -->\n",
       "<g id=\"node14\" class=\"node\">\n",
       "<title>289</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M595.5,-36C595.5,-36 565.5,-36 565.5,-36 559.5,-36 553.5,-30 553.5,-24 553.5,-24 553.5,-12 553.5,-12 553.5,-6 559.5,0 565.5,0 565.5,0 595.5,0 595.5,0 601.5,0 607.5,-6 607.5,-12 607.5,-12 607.5,-24 607.5,-24 607.5,-30 601.5,-36 595.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"580.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 227&#45;&gt;289 -->\n",
       "<g id=\"edge13\" class=\"edge\">\n",
       "<title>227&#45;&gt;289</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M568.8459,-71.9769C570.6962,-63.4071 572.6616,-54.3043 574.4408,-46.0638\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"577.8828,-46.7054 576.5722,-36.192 571.0405,-45.228 577.8828,-46.7054\"/>\n",
       "</g>\n",
       "<!-- 293 -->\n",
       "<g id=\"node16\" class=\"node\">\n",
       "<title>293</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M703.5,-36C703.5,-36 673.5,-36 673.5,-36 667.5,-36 661.5,-30 661.5,-24 661.5,-24 661.5,-12 661.5,-12 661.5,-6 667.5,0 673.5,0 673.5,0 703.5,0 703.5,0 709.5,0 715.5,-6 715.5,-12 715.5,-12 715.5,-24 715.5,-24 715.5,-30 709.5,-36 703.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"688.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 292&#45;&gt;293 -->\n",
       "<g id=\"edge15\" class=\"edge\">\n",
       "<title>292&#45;&gt;293</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M709.9681,-71.9769C706.4839,-63.2167 702.7782,-53.8995 699.4438,-45.5157\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"702.6834,-44.1905 695.7354,-36.192 696.179,-46.7775 702.6834,-44.1905\"/>\n",
       "</g>\n",
       "<!-- 320 -->\n",
       "<g id=\"node17\" class=\"node\">\n",
       "<title>320</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M775.5,-36C775.5,-36 745.5,-36 745.5,-36 739.5,-36 733.5,-30 733.5,-24 733.5,-24 733.5,-12 733.5,-12 733.5,-6 739.5,0 745.5,0 745.5,0 775.5,0 775.5,0 781.5,0 787.5,-6 787.5,-12 787.5,-12 787.5,-24 787.5,-24 787.5,-30 781.5,-36 775.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"760.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 292&#45;&gt;320 -->\n",
       "<g id=\"edge16\" class=\"edge\">\n",
       "<title>292&#45;&gt;320</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M737.8052,-71.9769C741.4884,-63.2167 745.4059,-53.8995 748.9309,-45.5157\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"752.2016,-46.7669 752.8511,-36.192 745.7487,-44.0537 752.2016,-46.7669\"/>\n",
       "</g>\n",
       "<!-- 368 -->\n",
       "<g id=\"node19\" class=\"node\">\n",
       "<title>368</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.937255\" stroke=\"#000000\" d=\"M1141.5,-244C1141.5,-244 1009.5,-244 1009.5,-244 1003.5,-244 997.5,-238 997.5,-232 997.5,-232 997.5,-188 997.5,-188 997.5,-182 1003.5,-176 1009.5,-176 1009.5,-176 1141.5,-176 1141.5,-176 1147.5,-176 1153.5,-182 1153.5,-188 1153.5,-188 1153.5,-232 1153.5,-232 1153.5,-238 1147.5,-244 1141.5,-244\"/>\n",
       "<text text-anchor=\"start\" x=\"1005.5\" y=\"-228.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">learning_rate_0.1 ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"1045\" y=\"-213.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"1030.5\" y=\"-198.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 180</text>\n",
       "<text text-anchor=\"start\" x=\"1034.5\" y=\"-183.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.673</text>\n",
       "</g>\n",
       "<!-- 367&#45;&gt;368 -->\n",
       "<g id=\"edge18\" class=\"edge\">\n",
       "<title>367&#45;&gt;368</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1154.1898,-279.9465C1143.7859,-270.6986 1132.411,-260.5876 1121.6243,-250.9994\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1123.6956,-248.1577 1113.8962,-244.13 1119.0451,-253.3896 1123.6956,-248.1577\"/>\n",
       "</g>\n",
       "<!-- 487 -->\n",
       "<g id=\"node26\" class=\"node\">\n",
       "<title>487</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.894118\" stroke=\"#000000\" d=\"M1375.5,-244C1375.5,-244 1243.5,-244 1243.5,-244 1237.5,-244 1231.5,-238 1231.5,-232 1231.5,-232 1231.5,-188 1231.5,-188 1231.5,-182 1237.5,-176 1243.5,-176 1243.5,-176 1375.5,-176 1375.5,-176 1381.5,-176 1387.5,-182 1387.5,-188 1387.5,-188 1387.5,-232 1387.5,-232 1387.5,-238 1381.5,-244 1375.5,-244\"/>\n",
       "<text text-anchor=\"start\" x=\"1239.5\" y=\"-228.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">learning_rate_0.3 ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"1279\" y=\"-213.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"1268.5\" y=\"-198.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 24</text>\n",
       "<text text-anchor=\"start\" x=\"1268.5\" y=\"-183.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.667</text>\n",
       "</g>\n",
       "<!-- 367&#45;&gt;487 -->\n",
       "<g id=\"edge25\" class=\"edge\">\n",
       "<title>367&#45;&gt;487</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1230.8102,-279.9465C1241.2141,-270.6986 1252.589,-260.5876 1263.3757,-250.9994\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1265.9549,-253.3896 1271.1038,-244.13 1261.3044,-248.1577 1265.9549,-253.3896\"/>\n",
       "</g>\n",
       "<!-- 369 -->\n",
       "<g id=\"node20\" class=\"node\">\n",
       "<title>369</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.925490\" stroke=\"#000000\" d=\"M953,-140C953,-140 822,-140 822,-140 816,-140 810,-134 810,-128 810,-128 810,-84 810,-84 810,-78 816,-72 822,-72 822,-72 953,-72 953,-72 959,-72 965,-78 965,-84 965,-84 965,-128 965,-128 965,-134 959,-140 953,-140\"/>\n",
       "<text text-anchor=\"start\" x=\"818\" y=\"-124.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">num_leaves_&#45;1.0 ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"857\" y=\"-109.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"846.5\" y=\"-94.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 90</text>\n",
       "<text text-anchor=\"start\" x=\"846.5\" y=\"-79.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.671</text>\n",
       "</g>\n",
       "<!-- 368&#45;&gt;369 -->\n",
       "<g id=\"edge19\" class=\"edge\">\n",
       "<title>368&#45;&gt;369</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1013.9418,-175.9465C996.0883,-166.0701 976.4552,-155.2093 958.0944,-145.0522\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"959.6411,-141.908 949.1965,-140.13 956.2527,-148.0333 959.6411,-141.908\"/>\n",
       "</g>\n",
       "<!-- 430 -->\n",
       "<g id=\"node23\" class=\"node\">\n",
       "<title>430</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.949020\" stroke=\"#000000\" d=\"M1155.5,-140C1155.5,-140 995.5,-140 995.5,-140 989.5,-140 983.5,-134 983.5,-128 983.5,-128 983.5,-84 983.5,-84 983.5,-78 989.5,-72 995.5,-72 995.5,-72 1155.5,-72 1155.5,-72 1161.5,-72 1167.5,-78 1167.5,-84 1167.5,-84 1167.5,-128 1167.5,-128 1167.5,-134 1161.5,-140 1155.5,-140\"/>\n",
       "<text text-anchor=\"start\" x=\"991.5\" y=\"-124.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">colsample_bytree_0.6 ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"1045\" y=\"-109.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"1034.5\" y=\"-94.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 90</text>\n",
       "<text text-anchor=\"start\" x=\"1034.5\" y=\"-79.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.675</text>\n",
       "</g>\n",
       "<!-- 368&#45;&gt;430 -->\n",
       "<g id=\"edge22\" class=\"edge\">\n",
       "<title>368&#45;&gt;430</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1075.5,-175.9465C1075.5,-167.776 1075.5,-158.9318 1075.5,-150.3697\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1079.0001,-150.13 1075.5,-140.13 1072.0001,-150.13 1079.0001,-150.13\"/>\n",
       "</g>\n",
       "<!-- 370 -->\n",
       "<g id=\"node21\" class=\"node\">\n",
       "<title>370</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M892.5,-36C892.5,-36 862.5,-36 862.5,-36 856.5,-36 850.5,-30 850.5,-24 850.5,-24 850.5,-12 850.5,-12 850.5,-6 856.5,0 862.5,0 862.5,0 892.5,0 892.5,0 898.5,0 904.5,-6 904.5,-12 904.5,-12 904.5,-24 904.5,-24 904.5,-30 898.5,-36 892.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"877.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 369&#45;&gt;370 -->\n",
       "<g id=\"edge20\" class=\"edge\">\n",
       "<title>369&#45;&gt;370</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M883.6337,-71.9769C882.6707,-63.5023 881.6485,-54.5065 880.7203,-46.3388\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"884.1741,-45.7328 879.5673,-36.192 877.2188,-46.5232 884.1741,-45.7328\"/>\n",
       "</g>\n",
       "<!-- 413 -->\n",
       "<g id=\"node22\" class=\"node\">\n",
       "<title>413</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M964.5,-36C964.5,-36 934.5,-36 934.5,-36 928.5,-36 922.5,-30 922.5,-24 922.5,-24 922.5,-12 922.5,-12 922.5,-6 928.5,0 934.5,0 934.5,0 964.5,0 964.5,0 970.5,0 976.5,-6 976.5,-12 976.5,-12 976.5,-24 976.5,-24 976.5,-30 970.5,-36 964.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"949.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 369&#45;&gt;413 -->\n",
       "<g id=\"edge21\" class=\"edge\">\n",
       "<title>369&#45;&gt;413</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M911.4708,-71.9769C917.9111,-62.8358 924.7787,-53.0883 930.8806,-44.4276\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"933.7846,-46.3826 936.6829,-36.192 928.0622,-42.3509 933.7846,-46.3826\"/>\n",
       "</g>\n",
       "<!-- 431 -->\n",
       "<g id=\"node24\" class=\"node\">\n",
       "<title>431</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M1063.5,-36C1063.5,-36 1033.5,-36 1033.5,-36 1027.5,-36 1021.5,-30 1021.5,-24 1021.5,-24 1021.5,-12 1021.5,-12 1021.5,-6 1027.5,0 1033.5,0 1033.5,0 1063.5,0 1063.5,0 1069.5,0 1075.5,-6 1075.5,-12 1075.5,-12 1075.5,-24 1075.5,-24 1075.5,-30 1069.5,-36 1063.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"1048.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 430&#45;&gt;431 -->\n",
       "<g id=\"edge23\" class=\"edge\">\n",
       "<title>430&#45;&gt;431</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1065.0611,-71.9769C1062.4025,-63.3119 1059.5767,-54.102 1057.0263,-45.7894\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1060.361,-44.7254 1054.0816,-36.192 1053.6689,-46.7787 1060.361,-44.7254\"/>\n",
       "</g>\n",
       "<!-- 484 -->\n",
       "<g id=\"node25\" class=\"node\">\n",
       "<title>484</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M1135.5,-36C1135.5,-36 1105.5,-36 1105.5,-36 1099.5,-36 1093.5,-30 1093.5,-24 1093.5,-24 1093.5,-12 1093.5,-12 1093.5,-6 1099.5,0 1105.5,0 1105.5,0 1135.5,0 1135.5,0 1141.5,0 1147.5,-6 1147.5,-12 1147.5,-12 1147.5,-24 1147.5,-24 1147.5,-30 1141.5,-36 1135.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"1120.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 430&#45;&gt;484 -->\n",
       "<g id=\"edge24\" class=\"edge\">\n",
       "<title>430&#45;&gt;484</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1092.8982,-71.9769C1097.4265,-63.1215 1102.2459,-53.6969 1106.5691,-45.2427\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1109.7606,-46.6889 1111.1973,-36.192 1103.5281,-43.5019 1109.7606,-46.6889\"/>\n",
       "</g>\n",
       "<!-- 488 -->\n",
       "<g id=\"node27\" class=\"node\">\n",
       "<title>488</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.854902\" stroke=\"#000000\" d=\"M1421.5,-140C1421.5,-140 1197.5,-140 1197.5,-140 1191.5,-140 1185.5,-134 1185.5,-128 1185.5,-128 1185.5,-84 1185.5,-84 1185.5,-78 1191.5,-72 1197.5,-72 1197.5,-72 1421.5,-72 1421.5,-72 1427.5,-72 1433.5,-78 1433.5,-84 1433.5,-84 1433.5,-128 1433.5,-128 1433.5,-134 1427.5,-140 1421.5,-140\"/>\n",
       "<text text-anchor=\"start\" x=\"1193.5\" y=\"-124.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">model_type_Lightgbm Dummies ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"1279\" y=\"-109.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"1268.5\" y=\"-94.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 12</text>\n",
       "<text text-anchor=\"start\" x=\"1268.5\" y=\"-79.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.661</text>\n",
       "</g>\n",
       "<!-- 487&#45;&gt;488 -->\n",
       "<g id=\"edge26\" class=\"edge\">\n",
       "<title>487&#45;&gt;488</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1309.5,-175.9465C1309.5,-167.776 1309.5,-158.9318 1309.5,-150.3697\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1313.0001,-150.13 1309.5,-140.13 1306.0001,-150.13 1313.0001,-150.13\"/>\n",
       "</g>\n",
       "<!-- 497 -->\n",
       "<g id=\"node30\" class=\"node\">\n",
       "<title>497</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.933333\" stroke=\"#000000\" d=\"M1623.5,-140C1623.5,-140 1463.5,-140 1463.5,-140 1457.5,-140 1451.5,-134 1451.5,-128 1451.5,-128 1451.5,-84 1451.5,-84 1451.5,-78 1457.5,-72 1463.5,-72 1463.5,-72 1623.5,-72 1623.5,-72 1629.5,-72 1635.5,-78 1635.5,-84 1635.5,-84 1635.5,-128 1635.5,-128 1635.5,-134 1629.5,-140 1623.5,-140\"/>\n",
       "<text text-anchor=\"start\" x=\"1459.5\" y=\"-124.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">colsample_bytree_0.8 ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"1513\" y=\"-109.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"1502.5\" y=\"-94.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 12</text>\n",
       "<text text-anchor=\"start\" x=\"1502.5\" y=\"-79.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.673</text>\n",
       "</g>\n",
       "<!-- 487&#45;&gt;497 -->\n",
       "<g id=\"edge29\" class=\"edge\">\n",
       "<title>487&#45;&gt;497</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1386.1204,-175.9465C1408.9483,-165.8008 1434.1138,-154.6161 1457.4987,-144.2228\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1458.9909,-147.3898 1466.7075,-140.13 1456.1479,-140.9931 1458.9909,-147.3898\"/>\n",
       "</g>\n",
       "<!-- 489 -->\n",
       "<g id=\"node28\" class=\"node\">\n",
       "<title>489</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M1288.5,-36C1288.5,-36 1258.5,-36 1258.5,-36 1252.5,-36 1246.5,-30 1246.5,-24 1246.5,-24 1246.5,-12 1246.5,-12 1246.5,-6 1252.5,0 1258.5,0 1258.5,0 1288.5,0 1288.5,0 1294.5,0 1300.5,-6 1300.5,-12 1300.5,-12 1300.5,-24 1300.5,-24 1300.5,-30 1294.5,-36 1288.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"1273.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 488&#45;&gt;489 -->\n",
       "<g id=\"edge27\" class=\"edge\">\n",
       "<title>488&#45;&gt;489</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1295.5815,-71.9769C1291.9977,-63.2167 1288.1862,-53.8995 1284.7564,-45.5157\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1287.968,-44.1222 1280.9422,-36.192 1281.4891,-46.7727 1287.968,-44.1222\"/>\n",
       "</g>\n",
       "<!-- 492 -->\n",
       "<g id=\"node29\" class=\"node\">\n",
       "<title>492</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M1360.5,-36C1360.5,-36 1330.5,-36 1330.5,-36 1324.5,-36 1318.5,-30 1318.5,-24 1318.5,-24 1318.5,-12 1318.5,-12 1318.5,-6 1324.5,0 1330.5,0 1330.5,0 1360.5,0 1360.5,0 1366.5,0 1372.5,-6 1372.5,-12 1372.5,-12 1372.5,-24 1372.5,-24 1372.5,-30 1366.5,-36 1360.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"1345.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 488&#45;&gt;492 -->\n",
       "<g id=\"edge28\" class=\"edge\">\n",
       "<title>488&#45;&gt;492</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1323.4185,-71.9769C1327.0023,-63.2167 1330.8138,-53.8995 1334.2436,-45.5157\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1337.5109,-46.7727 1338.0578,-36.192 1331.032,-44.1222 1337.5109,-46.7727\"/>\n",
       "</g>\n",
       "<!-- 498 -->\n",
       "<g id=\"node31\" class=\"node\">\n",
       "<title>498</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M1502.5,-36C1502.5,-36 1472.5,-36 1472.5,-36 1466.5,-36 1460.5,-30 1460.5,-24 1460.5,-24 1460.5,-12 1460.5,-12 1460.5,-6 1466.5,0 1472.5,0 1472.5,0 1502.5,0 1502.5,0 1508.5,0 1514.5,-6 1514.5,-12 1514.5,-12 1514.5,-24 1514.5,-24 1514.5,-30 1508.5,-36 1502.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"1487.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 497&#45;&gt;498 -->\n",
       "<g id=\"edge30\" class=\"edge\">\n",
       "<title>497&#45;&gt;498</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1521.8489,-71.9769C1516.0925,-62.931 1509.9581,-53.2913 1504.49,-44.6986\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1507.3983,-42.7495 1499.0767,-36.192 1501.4927,-46.5077 1507.3983,-42.7495\"/>\n",
       "</g>\n",
       "<!-- 503 -->\n",
       "<g id=\"node32\" class=\"node\">\n",
       "<title>503</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M1574.5,-36C1574.5,-36 1544.5,-36 1544.5,-36 1538.5,-36 1532.5,-30 1532.5,-24 1532.5,-24 1532.5,-12 1532.5,-12 1532.5,-6 1538.5,0 1544.5,0 1544.5,0 1574.5,0 1574.5,0 1580.5,0 1586.5,-6 1586.5,-12 1586.5,-12 1586.5,-24 1586.5,-24 1586.5,-30 1580.5,-36 1574.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"1559.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 497&#45;&gt;503 -->\n",
       "<g id=\"edge31\" class=\"edge\">\n",
       "<title>497&#45;&gt;503</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1549.686,-71.9769C1551.2442,-63.4071 1552.8992,-54.3043 1554.3975,-46.0638\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1557.847,-46.6568 1556.1924,-36.192 1550.9599,-45.4045 1557.847,-46.6568\"/>\n",
       "</g>\n",
       "<!-- 507 -->\n",
       "<g id=\"node34\" class=\"node\">\n",
       "<title>507</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.796078\" stroke=\"#000000\" d=\"M2096.5,-348C2096.5,-348 1886.5,-348 1886.5,-348 1880.5,-348 1874.5,-342 1874.5,-336 1874.5,-336 1874.5,-292 1874.5,-292 1874.5,-286 1880.5,-280 1886.5,-280 1886.5,-280 2096.5,-280 2096.5,-280 2102.5,-280 2108.5,-286 2108.5,-292 2108.5,-292 2108.5,-336 2108.5,-336 2108.5,-342 2102.5,-348 2096.5,-348\"/>\n",
       "<text text-anchor=\"start\" x=\"1882.5\" y=\"-332.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">model_type_Lightgbm Factors ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"1953.5\" y=\"-317.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.001</text>\n",
       "<text text-anchor=\"start\" x=\"1946.5\" y=\"-302.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 192</text>\n",
       "<text text-anchor=\"start\" x=\"1950.5\" y=\"-287.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.651</text>\n",
       "</g>\n",
       "<!-- 506&#45;&gt;507 -->\n",
       "<g id=\"edge33\" class=\"edge\">\n",
       "<title>506&#45;&gt;507</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1991.5,-383.9465C1991.5,-375.776 1991.5,-366.9318 1991.5,-358.3697\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1995.0001,-358.13 1991.5,-348.13 1988.0001,-358.13 1995.0001,-358.13\"/>\n",
       "</g>\n",
       "<!-- 668 -->\n",
       "<g id=\"node49\" class=\"node\">\n",
       "<title>668</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.470588\" stroke=\"#000000\" d=\"M2499.5,-348C2499.5,-348 2389.5,-348 2389.5,-348 2383.5,-348 2377.5,-342 2377.5,-336 2377.5,-336 2377.5,-292 2377.5,-292 2377.5,-286 2383.5,-280 2389.5,-280 2389.5,-280 2499.5,-280 2499.5,-280 2505.5,-280 2511.5,-286 2511.5,-292 2511.5,-292 2511.5,-336 2511.5,-336 2511.5,-342 2505.5,-348 2499.5,-348\"/>\n",
       "<text text-anchor=\"start\" x=\"2385.5\" y=\"-332.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">max_depth_3 ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"2406.5\" y=\"-317.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.001</text>\n",
       "<text text-anchor=\"start\" x=\"2399.5\" y=\"-302.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 144</text>\n",
       "<text text-anchor=\"start\" x=\"2403.5\" y=\"-287.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.598</text>\n",
       "</g>\n",
       "<!-- 506&#45;&gt;668 -->\n",
       "<g id=\"edge48\" class=\"edge\">\n",
       "<title>506&#45;&gt;668</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M2083.7173,-396.8287C2167.3209,-377.6349 2289.1259,-349.6709 2367.5968,-331.6555\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"2368.458,-335.0489 2377.4212,-329.4 2366.8916,-328.2264 2368.458,-335.0489\"/>\n",
       "</g>\n",
       "<!-- 508 -->\n",
       "<g id=\"node35\" class=\"node\">\n",
       "<title>508</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.725490\" stroke=\"#000000\" d=\"M1952.5,-244C1952.5,-244 1842.5,-244 1842.5,-244 1836.5,-244 1830.5,-238 1830.5,-232 1830.5,-232 1830.5,-188 1830.5,-188 1830.5,-182 1836.5,-176 1842.5,-176 1842.5,-176 1952.5,-176 1952.5,-176 1958.5,-176 1964.5,-182 1964.5,-188 1964.5,-188 1964.5,-232 1964.5,-232 1964.5,-238 1958.5,-244 1952.5,-244\"/>\n",
       "<text text-anchor=\"start\" x=\"1838.5\" y=\"-228.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">max_depth_3 ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"1859.5\" y=\"-213.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.001</text>\n",
       "<text text-anchor=\"start\" x=\"1852.5\" y=\"-198.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 114</text>\n",
       "<text text-anchor=\"start\" x=\"1856.5\" y=\"-183.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.639</text>\n",
       "</g>\n",
       "<!-- 507&#45;&gt;508 -->\n",
       "<g id=\"edge34\" class=\"edge\">\n",
       "<title>507&#45;&gt;508</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1960.7209,-279.9465C1952.5245,-270.8782 1943.578,-260.9799 1935.0625,-251.5585\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1937.6503,-249.2019 1928.3483,-244.13 1932.4571,-253.8957 1937.6503,-249.2019\"/>\n",
       "</g>\n",
       "<!-- 605 -->\n",
       "<g id=\"node42\" class=\"node\">\n",
       "<title>605</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.901961\" stroke=\"#000000\" d=\"M2166.5,-244C2166.5,-244 2006.5,-244 2006.5,-244 2000.5,-244 1994.5,-238 1994.5,-232 1994.5,-232 1994.5,-188 1994.5,-188 1994.5,-182 2000.5,-176 2006.5,-176 2006.5,-176 2166.5,-176 2166.5,-176 2172.5,-176 2178.5,-182 2178.5,-188 2178.5,-188 2178.5,-232 2178.5,-232 2178.5,-238 2172.5,-244 2166.5,-244\"/>\n",
       "<text text-anchor=\"start\" x=\"2002.5\" y=\"-228.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">colsample_bytree_0.8 ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"2056\" y=\"-213.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"2045.5\" y=\"-198.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 78</text>\n",
       "<text text-anchor=\"start\" x=\"2045.5\" y=\"-183.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.668</text>\n",
       "</g>\n",
       "<!-- 507&#45;&gt;605 -->\n",
       "<g id=\"edge41\" class=\"edge\">\n",
       "<title>507&#45;&gt;605</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M2022.6066,-279.9465C2030.8901,-270.8782 2039.9318,-260.9799 2048.5379,-251.5585\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"2051.1633,-253.8739 2055.3236,-244.13 2045.995,-249.1528 2051.1633,-253.8739\"/>\n",
       "</g>\n",
       "<!-- 509 -->\n",
       "<g id=\"node36\" class=\"node\">\n",
       "<title>509</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.760784\" stroke=\"#000000\" d=\"M1775.5,-140C1775.5,-140 1665.5,-140 1665.5,-140 1659.5,-140 1653.5,-134 1653.5,-128 1653.5,-128 1653.5,-84 1653.5,-84 1653.5,-78 1659.5,-72 1665.5,-72 1665.5,-72 1775.5,-72 1775.5,-72 1781.5,-72 1787.5,-78 1787.5,-84 1787.5,-84 1787.5,-128 1787.5,-128 1787.5,-134 1781.5,-140 1775.5,-140\"/>\n",
       "<text text-anchor=\"start\" x=\"1661.5\" y=\"-124.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">max_depth_5 ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"1682.5\" y=\"-109.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.001</text>\n",
       "<text text-anchor=\"start\" x=\"1675.5\" y=\"-94.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 100</text>\n",
       "<text text-anchor=\"start\" x=\"1679.5\" y=\"-79.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.645</text>\n",
       "</g>\n",
       "<!-- 508&#45;&gt;509 -->\n",
       "<g id=\"edge35\" class=\"edge\">\n",
       "<title>508&#45;&gt;509</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1839.5436,-175.9465C1822.8875,-166.1599 1804.5863,-155.4066 1787.4356,-145.3294\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1788.9816,-142.1783 1778.5866,-140.13 1785.4354,-148.2136 1788.9816,-142.1783\"/>\n",
       "</g>\n",
       "<!-- 594 -->\n",
       "<g id=\"node39\" class=\"node\">\n",
       "<title>594</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.466667\" stroke=\"#000000\" d=\"M1977.5,-140C1977.5,-140 1817.5,-140 1817.5,-140 1811.5,-140 1805.5,-134 1805.5,-128 1805.5,-128 1805.5,-84 1805.5,-84 1805.5,-78 1811.5,-72 1817.5,-72 1817.5,-72 1977.5,-72 1977.5,-72 1983.5,-72 1989.5,-78 1989.5,-84 1989.5,-84 1989.5,-128 1989.5,-128 1989.5,-134 1983.5,-140 1977.5,-140\"/>\n",
       "<text text-anchor=\"start\" x=\"1813.5\" y=\"-124.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">colsample_bytree_0.6 ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"1859.5\" y=\"-109.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.001</text>\n",
       "<text text-anchor=\"start\" x=\"1856.5\" y=\"-94.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 14</text>\n",
       "<text text-anchor=\"start\" x=\"1856.5\" y=\"-79.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.598</text>\n",
       "</g>\n",
       "<!-- 508&#45;&gt;594 -->\n",
       "<g id=\"edge38\" class=\"edge\">\n",
       "<title>508&#45;&gt;594</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1897.5,-175.9465C1897.5,-167.776 1897.5,-158.9318 1897.5,-150.3697\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1901.0001,-150.13 1897.5,-140.13 1894.0001,-150.13 1901.0001,-150.13\"/>\n",
       "</g>\n",
       "<!-- 510 -->\n",
       "<g id=\"node37\" class=\"node\">\n",
       "<title>510</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M1679.5,-36C1679.5,-36 1649.5,-36 1649.5,-36 1643.5,-36 1637.5,-30 1637.5,-24 1637.5,-24 1637.5,-12 1637.5,-12 1637.5,-6 1643.5,0 1649.5,0 1649.5,0 1679.5,0 1679.5,0 1685.5,0 1691.5,-6 1691.5,-12 1691.5,-12 1691.5,-24 1691.5,-24 1691.5,-30 1685.5,-36 1679.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"1664.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 509&#45;&gt;510 -->\n",
       "<g id=\"edge36\" class=\"edge\">\n",
       "<title>509&#45;&gt;510</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1698.8489,-71.9769C1693.0925,-62.931 1686.9581,-53.2913 1681.49,-44.6986\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1684.3983,-42.7495 1676.0767,-36.192 1678.4927,-46.5077 1684.3983,-42.7495\"/>\n",
       "</g>\n",
       "<!-- 583 -->\n",
       "<g id=\"node38\" class=\"node\">\n",
       "<title>583</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M1751.5,-36C1751.5,-36 1721.5,-36 1721.5,-36 1715.5,-36 1709.5,-30 1709.5,-24 1709.5,-24 1709.5,-12 1709.5,-12 1709.5,-6 1715.5,0 1721.5,0 1721.5,0 1751.5,0 1751.5,0 1757.5,0 1763.5,-6 1763.5,-12 1763.5,-12 1763.5,-24 1763.5,-24 1763.5,-30 1757.5,-36 1751.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"1736.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 509&#45;&gt;583 -->\n",
       "<g id=\"edge37\" class=\"edge\">\n",
       "<title>509&#45;&gt;583</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1726.686,-71.9769C1728.2442,-63.4071 1729.8992,-54.3043 1731.3975,-46.0638\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1734.847,-46.6568 1733.1924,-36.192 1727.9599,-45.4045 1734.847,-46.6568\"/>\n",
       "</g>\n",
       "<!-- 595 -->\n",
       "<g id=\"node40\" class=\"node\">\n",
       "<title>595</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M1862.5,-36C1862.5,-36 1832.5,-36 1832.5,-36 1826.5,-36 1820.5,-30 1820.5,-24 1820.5,-24 1820.5,-12 1820.5,-12 1820.5,-6 1826.5,0 1832.5,0 1832.5,0 1862.5,0 1862.5,0 1868.5,0 1874.5,-6 1874.5,-12 1874.5,-12 1874.5,-24 1874.5,-24 1874.5,-30 1868.5,-36 1862.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"1847.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 594&#45;&gt;595 -->\n",
       "<g id=\"edge39\" class=\"edge\">\n",
       "<title>594&#45;&gt;595</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1878.1687,-71.9769C1873.0831,-63.0262 1867.6671,-53.4941 1862.824,-44.9703\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1865.8196,-43.1575 1857.8363,-36.192 1859.7334,-46.6156 1865.8196,-43.1575\"/>\n",
       "</g>\n",
       "<!-- 602 -->\n",
       "<g id=\"node41\" class=\"node\">\n",
       "<title>602</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M1934.5,-36C1934.5,-36 1904.5,-36 1904.5,-36 1898.5,-36 1892.5,-30 1892.5,-24 1892.5,-24 1892.5,-12 1892.5,-12 1892.5,-6 1898.5,0 1904.5,0 1904.5,0 1934.5,0 1934.5,0 1940.5,0 1946.5,-6 1946.5,-12 1946.5,-12 1946.5,-24 1946.5,-24 1946.5,-30 1940.5,-36 1934.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"1919.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 594&#45;&gt;602 -->\n",
       "<g id=\"edge40\" class=\"edge\">\n",
       "<title>594&#45;&gt;602</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M1906.0058,-71.9769C1908.1482,-63.4071 1910.4239,-54.3043 1912.4841,-46.0638\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1915.9221,-46.7423 1914.952,-36.192 1909.1311,-45.0445 1915.9221,-46.7423\"/>\n",
       "</g>\n",
       "<!-- 606 -->\n",
       "<g id=\"node43\" class=\"node\">\n",
       "<title>606</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.933333\" stroke=\"#000000\" d=\"M2153.5,-140C2153.5,-140 2019.5,-140 2019.5,-140 2013.5,-140 2007.5,-134 2007.5,-128 2007.5,-128 2007.5,-84 2007.5,-84 2007.5,-78 2013.5,-72 2019.5,-72 2019.5,-72 2153.5,-72 2153.5,-72 2159.5,-72 2165.5,-78 2165.5,-84 2165.5,-84 2165.5,-128 2165.5,-128 2165.5,-134 2159.5,-140 2153.5,-140\"/>\n",
       "<text text-anchor=\"start\" x=\"2015.5\" y=\"-124.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">num_leaves_32.0 ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"2056\" y=\"-109.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"2045.5\" y=\"-94.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 24</text>\n",
       "<text text-anchor=\"start\" x=\"2045.5\" y=\"-79.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.673</text>\n",
       "</g>\n",
       "<!-- 605&#45;&gt;606 -->\n",
       "<g id=\"edge42\" class=\"edge\">\n",
       "<title>605&#45;&gt;606</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M2086.5,-175.9465C2086.5,-167.776 2086.5,-158.9318 2086.5,-150.3697\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"2090.0001,-150.13 2086.5,-140.13 2083.0001,-150.13 2090.0001,-150.13\"/>\n",
       "</g>\n",
       "<!-- 625 -->\n",
       "<g id=\"node46\" class=\"node\">\n",
       "<title>625</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.886275\" stroke=\"#000000\" d=\"M2305.5,-140C2305.5,-140 2195.5,-140 2195.5,-140 2189.5,-140 2183.5,-134 2183.5,-128 2183.5,-128 2183.5,-84 2183.5,-84 2183.5,-78 2189.5,-72 2195.5,-72 2195.5,-72 2305.5,-72 2305.5,-72 2311.5,-72 2317.5,-78 2317.5,-84 2317.5,-84 2317.5,-128 2317.5,-128 2317.5,-134 2311.5,-140 2305.5,-140\"/>\n",
       "<text text-anchor=\"start\" x=\"2191.5\" y=\"-124.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">max_depth_6 ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"2220\" y=\"-109.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"2209.5\" y=\"-94.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 54</text>\n",
       "<text text-anchor=\"start\" x=\"2209.5\" y=\"-79.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.665</text>\n",
       "</g>\n",
       "<!-- 605&#45;&gt;625 -->\n",
       "<g id=\"edge45\" class=\"edge\">\n",
       "<title>605&#45;&gt;625</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M2140.1997,-175.9465C2155.4909,-166.2497 2172.2785,-155.6039 2188.0429,-145.6069\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"2190.1089,-148.4412 2196.6796,-140.13 2186.3601,-142.5297 2190.1089,-148.4412\"/>\n",
       "</g>\n",
       "<!-- 607 -->\n",
       "<g id=\"node44\" class=\"node\">\n",
       "<title>607</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M2039.5,-36C2039.5,-36 2009.5,-36 2009.5,-36 2003.5,-36 1997.5,-30 1997.5,-24 1997.5,-24 1997.5,-12 1997.5,-12 1997.5,-6 2003.5,0 2009.5,0 2009.5,0 2039.5,0 2039.5,0 2045.5,0 2051.5,-6 2051.5,-12 2051.5,-12 2051.5,-24 2051.5,-24 2051.5,-30 2045.5,-36 2039.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"2024.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 606&#45;&gt;607 -->\n",
       "<g id=\"edge43\" class=\"edge\">\n",
       "<title>606&#45;&gt;607</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M2062.5292,-71.9769C2056.0889,-62.8358 2049.2213,-53.0883 2043.1194,-44.4276\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"2045.9378,-42.3509 2037.3171,-36.192 2040.2154,-46.3826 2045.9378,-42.3509\"/>\n",
       "</g>\n",
       "<!-- 622 -->\n",
       "<g id=\"node45\" class=\"node\">\n",
       "<title>622</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M2111.5,-36C2111.5,-36 2081.5,-36 2081.5,-36 2075.5,-36 2069.5,-30 2069.5,-24 2069.5,-24 2069.5,-12 2069.5,-12 2069.5,-6 2075.5,0 2081.5,0 2081.5,0 2111.5,0 2111.5,0 2117.5,0 2123.5,-6 2123.5,-12 2123.5,-12 2123.5,-24 2123.5,-24 2123.5,-30 2117.5,-36 2111.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"2096.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 606&#45;&gt;622 -->\n",
       "<g id=\"edge44\" class=\"edge\">\n",
       "<title>606&#45;&gt;622</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M2090.3663,-71.9769C2091.3293,-63.5023 2092.3515,-54.5065 2093.2797,-46.3388\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"2096.7812,-46.5232 2094.4327,-36.192 2089.8259,-45.7328 2096.7812,-46.5232\"/>\n",
       "</g>\n",
       "<!-- 626 -->\n",
       "<g id=\"node47\" class=\"node\">\n",
       "<title>626</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M2197.5,-36C2197.5,-36 2167.5,-36 2167.5,-36 2161.5,-36 2155.5,-30 2155.5,-24 2155.5,-24 2155.5,-12 2155.5,-12 2155.5,-6 2161.5,0 2167.5,0 2167.5,0 2197.5,0 2197.5,0 2203.5,0 2209.5,-6 2209.5,-12 2209.5,-12 2209.5,-24 2209.5,-24 2209.5,-30 2203.5,-36 2197.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"2182.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 625&#45;&gt;626 -->\n",
       "<g id=\"edge46\" class=\"edge\">\n",
       "<title>625&#45;&gt;626</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M2224.2094,-71.9769C2217.0723,-62.7406 2209.4567,-52.8851 2202.7125,-44.1573\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"2205.4414,-41.9647 2196.5574,-36.192 2199.9024,-46.2449 2205.4414,-41.9647\"/>\n",
       "</g>\n",
       "<!-- 665 -->\n",
       "<g id=\"node48\" class=\"node\">\n",
       "<title>665</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M2269.5,-36C2269.5,-36 2239.5,-36 2239.5,-36 2233.5,-36 2227.5,-30 2227.5,-24 2227.5,-24 2227.5,-12 2227.5,-12 2227.5,-6 2233.5,0 2239.5,0 2239.5,0 2269.5,0 2269.5,0 2275.5,0 2281.5,-6 2281.5,-12 2281.5,-12 2281.5,-24 2281.5,-24 2281.5,-30 2275.5,-36 2269.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"2254.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 625&#45;&gt;665 -->\n",
       "<g id=\"edge47\" class=\"edge\">\n",
       "<title>625&#45;&gt;665</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M2252.0465,-71.9769C2252.4317,-63.5023 2252.8406,-54.5065 2253.2119,-46.3388\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"2256.7153,-46.3406 2253.6731,-36.192 2249.7225,-46.0227 2256.7153,-46.3406\"/>\n",
       "</g>\n",
       "<!-- 669 -->\n",
       "<g id=\"node50\" class=\"node\">\n",
       "<title>669</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.501961\" stroke=\"#000000\" d=\"M2549.5,-244C2549.5,-244 2339.5,-244 2339.5,-244 2333.5,-244 2327.5,-238 2327.5,-232 2327.5,-232 2327.5,-188 2327.5,-188 2327.5,-182 2333.5,-176 2339.5,-176 2339.5,-176 2549.5,-176 2549.5,-176 2555.5,-176 2561.5,-182 2561.5,-188 2561.5,-188 2561.5,-232 2561.5,-232 2561.5,-238 2555.5,-244 2549.5,-244\"/>\n",
       "<text text-anchor=\"start\" x=\"2335.5\" y=\"-228.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">model_type_Lightgbm Factors ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"2406.5\" y=\"-213.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.001</text>\n",
       "<text text-anchor=\"start\" x=\"2399.5\" y=\"-198.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 130</text>\n",
       "<text text-anchor=\"start\" x=\"2403.5\" y=\"-183.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.604</text>\n",
       "</g>\n",
       "<!-- 668&#45;&gt;669 -->\n",
       "<g id=\"edge49\" class=\"edge\">\n",
       "<title>668&#45;&gt;669</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M2444.5,-279.9465C2444.5,-271.776 2444.5,-262.9318 2444.5,-254.3697\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"2448.0001,-254.13 2444.5,-244.13 2441.0001,-254.13 2448.0001,-254.13\"/>\n",
       "</g>\n",
       "<!-- 772 -->\n",
       "<g id=\"node57\" class=\"node\">\n",
       "<title>772</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.149020\" stroke=\"#000000\" d=\"M2844.5,-244C2844.5,-244 2634.5,-244 2634.5,-244 2628.5,-244 2622.5,-238 2622.5,-232 2622.5,-232 2622.5,-188 2622.5,-188 2622.5,-182 2628.5,-176 2634.5,-176 2634.5,-176 2844.5,-176 2844.5,-176 2850.5,-176 2856.5,-182 2856.5,-188 2856.5,-188 2856.5,-232 2856.5,-232 2856.5,-238 2850.5,-244 2844.5,-244\"/>\n",
       "<text text-anchor=\"start\" x=\"2630.5\" y=\"-228.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">model_type_Lightgbm Factors ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"2709\" y=\"-213.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"2698.5\" y=\"-198.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 14</text>\n",
       "<text text-anchor=\"start\" x=\"2698.5\" y=\"-183.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.547</text>\n",
       "</g>\n",
       "<!-- 668&#45;&gt;772 -->\n",
       "<g id=\"edge56\" class=\"edge\">\n",
       "<title>668&#45;&gt;772</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M2511.8092,-290.2707C2547.7033,-277.6165 2592.7002,-261.7531 2633.1938,-247.4774\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"2634.5716,-250.7029 2642.839,-244.0771 2632.2441,-244.1012 2634.5716,-250.7029\"/>\n",
       "</g>\n",
       "<!-- 670 -->\n",
       "<g id=\"node51\" class=\"node\">\n",
       "<title>670</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.443137\" stroke=\"#000000\" d=\"M2457.5,-140C2457.5,-140 2347.5,-140 2347.5,-140 2341.5,-140 2335.5,-134 2335.5,-128 2335.5,-128 2335.5,-84 2335.5,-84 2335.5,-78 2341.5,-72 2347.5,-72 2347.5,-72 2457.5,-72 2457.5,-72 2463.5,-72 2469.5,-78 2469.5,-84 2469.5,-84 2469.5,-128 2469.5,-128 2469.5,-134 2463.5,-140 2457.5,-140\"/>\n",
       "<text text-anchor=\"start\" x=\"2343.5\" y=\"-124.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">max_depth_5 ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"2364.5\" y=\"-109.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.001</text>\n",
       "<text text-anchor=\"start\" x=\"2361.5\" y=\"-94.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 80</text>\n",
       "<text text-anchor=\"start\" x=\"2361.5\" y=\"-79.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.594</text>\n",
       "</g>\n",
       "<!-- 669&#45;&gt;670 -->\n",
       "<g id=\"edge50\" class=\"edge\">\n",
       "<title>669&#45;&gt;670</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M2430.7476,-175.9465C2427.3392,-167.5067 2423.6405,-158.3479 2420.0769,-149.5239\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"2423.2733,-148.0918 2416.2833,-140.13 2416.7826,-150.7131 2423.2733,-148.0918\"/>\n",
       "</g>\n",
       "<!-- 735 -->\n",
       "<g id=\"node54\" class=\"node\">\n",
       "<title>735</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.603922\" stroke=\"#000000\" d=\"M2633.5,-140C2633.5,-140 2499.5,-140 2499.5,-140 2493.5,-140 2487.5,-134 2487.5,-128 2487.5,-128 2487.5,-84 2487.5,-84 2487.5,-78 2493.5,-72 2499.5,-72 2499.5,-72 2633.5,-72 2633.5,-72 2639.5,-72 2645.5,-78 2645.5,-84 2645.5,-84 2645.5,-128 2645.5,-128 2645.5,-134 2639.5,-140 2633.5,-140\"/>\n",
       "<text text-anchor=\"start\" x=\"2495.5\" y=\"-124.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">num_leaves_32.0 ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"2536\" y=\"-109.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"2525.5\" y=\"-94.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 50</text>\n",
       "<text text-anchor=\"start\" x=\"2529\" y=\"-79.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.62</text>\n",
       "</g>\n",
       "<!-- 669&#45;&gt;735 -->\n",
       "<g id=\"edge53\" class=\"edge\">\n",
       "<title>669&#45;&gt;735</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M2484.4474,-175.9465C2495.4012,-166.6088 2507.3873,-156.3911 2518.732,-146.7203\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"2521.1233,-149.2809 2526.4629,-140.13 2516.5821,-143.9538 2521.1233,-149.2809\"/>\n",
       "</g>\n",
       "<!-- 671 -->\n",
       "<g id=\"node52\" class=\"node\">\n",
       "<title>671</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M2355.5,-36C2355.5,-36 2325.5,-36 2325.5,-36 2319.5,-36 2313.5,-30 2313.5,-24 2313.5,-24 2313.5,-12 2313.5,-12 2313.5,-6 2319.5,0 2325.5,0 2325.5,0 2355.5,0 2355.5,0 2361.5,0 2367.5,-6 2367.5,-12 2367.5,-12 2367.5,-24 2367.5,-24 2367.5,-30 2361.5,-36 2355.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"2340.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 670&#45;&gt;671 -->\n",
       "<g id=\"edge51\" class=\"edge\">\n",
       "<title>670&#45;&gt;671</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M2378.5292,-71.9769C2372.0889,-62.8358 2365.2213,-53.0883 2359.1194,-44.4276\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"2361.9378,-42.3509 2353.3171,-36.192 2356.2154,-46.3826 2361.9378,-42.3509\"/>\n",
       "</g>\n",
       "<!-- 730 -->\n",
       "<g id=\"node53\" class=\"node\">\n",
       "<title>730</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M2427.5,-36C2427.5,-36 2397.5,-36 2397.5,-36 2391.5,-36 2385.5,-30 2385.5,-24 2385.5,-24 2385.5,-12 2385.5,-12 2385.5,-6 2391.5,0 2397.5,0 2397.5,0 2427.5,0 2427.5,0 2433.5,0 2439.5,-6 2439.5,-12 2439.5,-12 2439.5,-24 2439.5,-24 2439.5,-30 2433.5,-36 2427.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"2412.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 670&#45;&gt;730 -->\n",
       "<g id=\"edge52\" class=\"edge\">\n",
       "<title>670&#45;&gt;730</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M2406.3663,-71.9769C2407.3293,-63.5023 2408.3515,-54.5065 2409.2797,-46.3388\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"2412.7812,-46.5232 2410.4327,-36.192 2405.8259,-45.7328 2412.7812,-46.5232\"/>\n",
       "</g>\n",
       "<!-- 736 -->\n",
       "<g id=\"node55\" class=\"node\">\n",
       "<title>736</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M2523.5,-36C2523.5,-36 2493.5,-36 2493.5,-36 2487.5,-36 2481.5,-30 2481.5,-24 2481.5,-24 2481.5,-12 2481.5,-12 2481.5,-6 2487.5,0 2493.5,0 2493.5,0 2523.5,0 2523.5,0 2529.5,0 2535.5,-6 2535.5,-12 2535.5,-12 2535.5,-24 2535.5,-24 2535.5,-30 2529.5,-36 2523.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"2508.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 735&#45;&gt;736 -->\n",
       "<g id=\"edge54\" class=\"edge\">\n",
       "<title>735&#45;&gt;736</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M2544.0757,-71.9769C2538.1136,-62.931 2531.7602,-53.2913 2526.0968,-44.6986\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"2528.9157,-42.6154 2520.4902,-36.192 2523.071,-46.4677 2528.9157,-42.6154\"/>\n",
       "</g>\n",
       "<!-- 769 -->\n",
       "<g id=\"node56\" class=\"node\">\n",
       "<title>769</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M2595.5,-36C2595.5,-36 2565.5,-36 2565.5,-36 2559.5,-36 2553.5,-30 2553.5,-24 2553.5,-24 2553.5,-12 2553.5,-12 2553.5,-6 2559.5,0 2565.5,0 2565.5,0 2595.5,0 2595.5,0 2601.5,0 2607.5,-6 2607.5,-12 2607.5,-12 2607.5,-24 2607.5,-24 2607.5,-30 2601.5,-36 2595.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"2580.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 735&#45;&gt;769 -->\n",
       "<g id=\"edge55\" class=\"edge\">\n",
       "<title>735&#45;&gt;769</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M2571.9128,-71.9769C2573.261,-63.5023 2574.6921,-54.5065 2575.9916,-46.3388\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"2579.4911,-46.6177 2577.6058,-36.192 2572.5781,-45.5178 2579.4911,-46.6177\"/>\n",
       "</g>\n",
       "<!-- 773 -->\n",
       "<g id=\"node58\" class=\"node\">\n",
       "<title>773</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.203922\" stroke=\"#000000\" d=\"M2803,-140C2803,-140 2676,-140 2676,-140 2670,-140 2664,-134 2664,-128 2664,-128 2664,-84 2664,-84 2664,-78 2670,-72 2676,-72 2676,-72 2803,-72 2803,-72 2809,-72 2815,-78 2815,-84 2815,-84 2815,-128 2815,-128 2815,-134 2809,-140 2803,-140\"/>\n",
       "<text text-anchor=\"start\" x=\"2672\" y=\"-124.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">num_leaves_8.0 ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"2709\" y=\"-109.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"2698.5\" y=\"-94.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 10</text>\n",
       "<text text-anchor=\"start\" x=\"2698.5\" y=\"-79.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.556</text>\n",
       "</g>\n",
       "<!-- 772&#45;&gt;773 -->\n",
       "<g id=\"edge57\" class=\"edge\">\n",
       "<title>772&#45;&gt;773</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M2739.5,-175.9465C2739.5,-167.776 2739.5,-158.9318 2739.5,-150.3697\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"2743.0001,-150.13 2739.5,-140.13 2736.0001,-150.13 2743.0001,-150.13\"/>\n",
       "</g>\n",
       "<!-- 778 -->\n",
       "<g id=\"node61\" class=\"node\">\n",
       "<title>778</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.011765\" stroke=\"#000000\" d=\"M2957.5,-140C2957.5,-140 2845.5,-140 2845.5,-140 2839.5,-140 2833.5,-134 2833.5,-128 2833.5,-128 2833.5,-84 2833.5,-84 2833.5,-78 2839.5,-72 2845.5,-72 2845.5,-72 2957.5,-72 2957.5,-72 2963.5,-72 2969.5,-78 2969.5,-84 2969.5,-84 2969.5,-128 2969.5,-128 2969.5,-134 2963.5,-140 2957.5,-140\"/>\n",
       "<text text-anchor=\"start\" x=\"2841.5\" y=\"-124.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">boosting_gbdt ≤ 0.5</text>\n",
       "<text text-anchor=\"start\" x=\"2871\" y=\"-109.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"2864\" y=\"-94.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 4</text>\n",
       "<text text-anchor=\"start\" x=\"2860.5\" y=\"-79.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.525</text>\n",
       "</g>\n",
       "<!-- 772&#45;&gt;778 -->\n",
       "<g id=\"edge60\" class=\"edge\">\n",
       "<title>772&#45;&gt;778</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M2792.5449,-175.9465C2807.6495,-166.2497 2824.2324,-155.6039 2839.8046,-145.6069\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"2841.8116,-148.4777 2848.336,-140.13 2838.03,-142.5871 2841.8116,-148.4777\"/>\n",
       "</g>\n",
       "<!-- 774 -->\n",
       "<g id=\"node59\" class=\"node\">\n",
       "<title>774</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M2709.5,-36C2709.5,-36 2679.5,-36 2679.5,-36 2673.5,-36 2667.5,-30 2667.5,-24 2667.5,-24 2667.5,-12 2667.5,-12 2667.5,-6 2673.5,0 2679.5,0 2679.5,0 2709.5,0 2709.5,0 2715.5,0 2721.5,-6 2721.5,-12 2721.5,-12 2721.5,-24 2721.5,-24 2721.5,-30 2715.5,-36 2709.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"2694.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 773&#45;&gt;774 -->\n",
       "<g id=\"edge58\" class=\"edge\">\n",
       "<title>773&#45;&gt;774</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M2722.1018,-71.9769C2717.5735,-63.1215 2712.7541,-53.6969 2708.4309,-45.2427\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"2711.4719,-43.5019 2703.8027,-36.192 2705.2394,-46.6889 2711.4719,-43.5019\"/>\n",
       "</g>\n",
       "<!-- 775 -->\n",
       "<g id=\"node60\" class=\"node\">\n",
       "<title>775</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M2781.5,-36C2781.5,-36 2751.5,-36 2751.5,-36 2745.5,-36 2739.5,-30 2739.5,-24 2739.5,-24 2739.5,-12 2739.5,-12 2739.5,-6 2745.5,0 2751.5,0 2751.5,0 2781.5,0 2781.5,0 2787.5,0 2793.5,-6 2793.5,-12 2793.5,-12 2793.5,-24 2793.5,-24 2793.5,-30 2787.5,-36 2781.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"2766.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 773&#45;&gt;775 -->\n",
       "<g id=\"edge59\" class=\"edge\">\n",
       "<title>773&#45;&gt;775</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M2749.9389,-71.9769C2752.5975,-63.3119 2755.4233,-54.102 2757.9737,-45.7894\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"2761.3311,-46.7787 2760.9184,-36.192 2754.639,-44.7254 2761.3311,-46.7787\"/>\n",
       "</g>\n",
       "<!-- 779 -->\n",
       "<g id=\"node62\" class=\"node\">\n",
       "<title>779</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M2880.5,-36C2880.5,-36 2850.5,-36 2850.5,-36 2844.5,-36 2838.5,-30 2838.5,-24 2838.5,-24 2838.5,-12 2838.5,-12 2838.5,-6 2844.5,0 2850.5,0 2850.5,0 2880.5,0 2880.5,0 2886.5,0 2892.5,-6 2892.5,-12 2892.5,-12 2892.5,-24 2892.5,-24 2892.5,-30 2886.5,-36 2880.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"2865.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 778&#45;&gt;779 -->\n",
       "<g id=\"edge61\" class=\"edge\">\n",
       "<title>778&#45;&gt;779</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M2887.5815,-71.9769C2883.9977,-63.2167 2880.1862,-53.8995 2876.7564,-45.5157\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"2879.968,-44.1222 2872.9422,-36.192 2873.4891,-46.7727 2879.968,-44.1222\"/>\n",
       "</g>\n",
       "<!-- 780 -->\n",
       "<g id=\"node63\" class=\"node\">\n",
       "<title>780</title>\n",
       "<path fill=\"#c0c0c0\" stroke=\"#000000\" d=\"M2952.5,-36C2952.5,-36 2922.5,-36 2922.5,-36 2916.5,-36 2910.5,-30 2910.5,-24 2910.5,-24 2910.5,-12 2910.5,-12 2910.5,-6 2916.5,0 2922.5,0 2922.5,0 2952.5,0 2952.5,0 2958.5,0 2964.5,-6 2964.5,-12 2964.5,-12 2964.5,-24 2964.5,-24 2964.5,-30 2958.5,-36 2952.5,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"2937.5\" y=\"-14.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">(...)</text>\n",
       "</g>\n",
       "<!-- 778&#45;&gt;780 -->\n",
       "<g id=\"edge62\" class=\"edge\">\n",
       "<title>778&#45;&gt;780</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M2915.4185,-71.9769C2919.0023,-63.2167 2922.8138,-53.8995 2926.2436,-45.5157\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"2929.5109,-46.7727 2930.0578,-36.192 2923.032,-44.1222 2929.5109,-46.7727\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.files.Source at 0x7f0d86716710>"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# out_file = 'model_tuning/xgb_tree.dot'\n",
    "out_file = None\n",
    "dot_data = export_graphviz(reg_tree,\n",
    "                          out_file=out_file,\n",
    "                          feature_names=features,\n",
    "                          max_depth=4,\n",
    "                          filled=True,\n",
    "                          rounded=True,\n",
    "                          special_characters=True)\n",
    "if out_file is not None:\n",
    "    dot_data = Path(out_file).read_text()\n",
    "\n",
    "graphviz.Source(dot_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-24T17:37:39.005273Z",
     "start_time": "2018-10-24T17:37:39.000228Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "learning_rate_0.01             0.435011\n",
       "colsample_bytree_1.0           0.250942\n",
       "max_depth_3                    0.077323\n",
       "model_type_Lightgbm Factors    0.070071\n",
       "max_depth_5                    0.041259\n",
       "colsample_bytree_0.6           0.027643\n",
       "num_leaves_16.0                0.014961\n",
       "max_depth_4                    0.009869\n",
       "max_depth_7                    0.008298\n",
       "boosting_dart                  0.007767\n",
       "dtype: float64"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(reg_tree.feature_importances_, index=features).sort_values(ascending=False).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## XGB Parameter Usage"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following plot shows the effect of different max_depth settings on the validation score for the LightGBM and XGBoost models: shallower trees produce a wider range of outcomes and need to be combined with appropriate learning rates and regularization settings to produce the strong result shown in the preceding table:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-23T02:09:43.524959Z",
     "start_time": "2018-10-23T02:09:43.521274Z"
    }
   },
   "outputs": [],
   "source": [
    "reg_tree = DecisionTreeRegressor(criterion='mse',\n",
    "                                 splitter='best',\n",
    "                                 max_depth=4,\n",
    "                                 min_samples_split=5,\n",
    "                                 min_samples_leaf=10,\n",
    "                                 min_weight_fraction_leaf=0.0,\n",
    "                                 max_features=None,\n",
    "                                 random_state=42,\n",
    "                                 max_leaf_nodes=None,\n",
    "                                 min_impurity_decrease=0.0,\n",
    "                                 min_impurity_split=None,\n",
    "                                 presort=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeRegressor(criterion='mse', max_depth=4, max_features=None,\n",
       "           max_leaf_nodes=None, min_impurity_decrease=0.0,\n",
       "           min_impurity_split=None, min_samples_leaf=10,\n",
       "           min_samples_split=5, min_weight_fraction_leaf=0.0,\n",
       "           presort=False, random_state=42, splitter='best')"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_score = lgb_xgb_combined.valid\n",
    "feature_data = lgb_xgb_combined.drop(['train', 'valid', 'time', 'rounds'], axis=1).fillna(-1)\n",
    "feature_data = pd.get_dummies(feature_data, columns=feature_data.columns)\n",
    "features = feature_data.columns\n",
    "reg_tree.fit(X=feature_data, y=test_score)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-23T02:10:05.575770Z",
     "start_time": "2018-10-23T02:10:05.536545Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeRegressor(criterion='mse', max_depth=4, max_features=None,\n",
       "           max_leaf_nodes=None, min_impurity_decrease=0.0,\n",
       "           min_impurity_split=None, min_samples_leaf=10,\n",
       "           min_samples_split=5, min_weight_fraction_leaf=0.0,\n",
       "           presort=False, random_state=42, splitter='best')"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xgb_data = lgb_xgb_combined[(lgb_xgb_combined.model_type=='Xgb Dummies')]\n",
    "xgb_data.boosting = pd.factorize(xgb_data.boosting)[0]\n",
    "xgb_features = ['boosting', 'learning_rate', 'max_depth', 'colsample_bytree', 'min_gain_to_split']\n",
    "reg_tree.fit(X=xgb_data[xgb_features], y=xgb_data.valid)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-23T02:10:07.247403Z",
     "start_time": "2018-10-23T02:10:07.084847Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbYAAAD7CAYAAAAGuLF2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAH0pJREFUeJzt3Xl8TWfix/HPTXJjSZAItf5aEbtKUVpTRVvaWrq8aG1pQ7RlTNG+2hqxtTQ09qral6lYI5a080JrLO2o6rSMpdRcYqmiWkEsCW62e35/zE9+Y0ISKk7u4/v+i3PPeZ7vOS7fe27ufTgsy7IQERExhI/dAURERG4nFZuIiBhFxSYiIkZRsYmIiFFUbCIiYhQVm4iIGEXFJiIiRlGxiYiIUVRsIiJiFD+7A9yNdu7cSYkSJeyOcUvS09MpVqyY3TFumTfnV3b7eHN+k7Knp6fTsGHDfI9TsdnA4XBQt25du2PcEpfL5bXZwbvzK7t9vDm/SdldLleBjtNbkSIiYhQVm4iIGEXFJiIiRlGxiYiIUVRsIiJiFBWbiIgYRcUmIiJGUbGJiIhRVGwiImIUFZsN/P397Y5wy7x1BYOrvDm/stvHm/MXpezuzOw7Mo+W1LKBj48P1QavtTuGiMgddXRshzsyj+7YRETEKCo2ERExiopNRESMomITERGjqNhERMQoKjYRETHKHS+2xMREJk6ceNvH7d+//20fMy/bt29n//79d3ROERHJnzF3bNOmTbuj861atYrk5OQ7OqeIiOTPti9oL1q0iDVr1uBwOGjfvj09evQgKSmJsWPH4vF4uHjxIsOHD6dx48Y8/vjjVK9enerVq5Oamoq/vz+//PILycnJjB07lvr169O8eXO2bt1KZGQkderU4eDBg6SlpTFlyhSqVKnC9OnT2bhxI2XLluXKlSu8+eabPPzww9fN9swzz1CtWjX8/f0ZNGgQI0eOJD09nfPnz9OvXz8qVqzIli1b2LdvHzVq1OCHH34gLi4OHx8fHnzwQQYOHHiHr6aIiFxlS7EdP36cHTt2sHTpUhwOB1FRUTz66KMcOnSI6OhoateuzerVq0lMTKRx48b8+uuvJCYmEhwczODBg6lcuTIxMTEsX76chIQEYmJirhk/PDycYcOGMXnyZNauXUvLli3ZsmULK1euJDMzk2effTbPfJcvX+b111+nXr16fPvtt/Tq1YuHH36YnTt3MnXqVObPn0+LFi1o3749JUuWZOrUqaxatYoSJUrw5z//ma1bt9K8efPCvIQiIl7J5XIVeF+3231T+19lS7H9+OOPZGVlERUVBcCFCxc4duwY99xzDzNmzKB48eJcunSJwMBAAIKDgwkODs45/uraZxUrVmTnzp25xq9Xr17O42fOnOHw4cM0aNAAX19ffH19uf/++/PNGBoaCkD58uWZOXMmK1euxOFwkJWVdc1+x44dIyUlhT59+gBw6dIljh8/fpNXRETk7nAza1e6XK5r9i9oydlSbHXq1MHtdjNv3jwcDgdxcXHUqlWLfv36MXHiRMLCwvj444/55ZdfgH+vrfifHA7HTc1Xo0YNFi1ahMfjISsri3/961/5HnN1zilTptC5c2datWrFqlWr+PTTT3MyWJZF1apVqVSpEp988glOp5PExMQiteioiMjdxpZiCw0NJSgoiO7du5ORkUF4eDgVKlTgueee4/XXXyckJISKFSty7ty52zJf7dq1adWqFV26dCE4OBin04mfX8FOvW3btnzwwQfMnj2bSpUq5WR64IEHmDhxIh999BFRUVFERkaSnZ1NlSpVaNeu3W3JLSIiN89hWZZld4jCdvbsWdatW8dLL71ERkYGHTp0YMGCBVSuXNmWPC6Xi3YLjtgyt4iIXW52df/rvRVZkHfE7or/tiY4OJgff/yRF154AYfDQefOnTlz5gzR0dG59m3Xrh0RERE2pBQRkdvhrig2Hx8fxowZk2v7okWLbEgjIiKFyZgvaIuIiICKTUREDKNiExERo6jYRETEKHfFh0eKGo/Hc9MfexUR8XbuzGyKO30LfR7dsdkgIyPD7gi37FbWbStKvDm/stvHm/MXpex3otRAxSYiIoZRsYmIiFFUbCIiYhQVm4iIGEXFJiIiRlGxiYiIUVRsIiJiFBWbiIgYRcUmIiJGUbGJiIhRVGwiImIUFZuIiBhFxSYiIkZRsYmIiFFUbCIiYhQVm4iIGEXFJiIiRlGxiYiIUVRsIiJiFBWbiIgYRcVmA39/f7sj3LK6devaHeF3ubdadbsjiEgh87M7wN3Ix8eHaoPX2h3jrnR0bAe7I4hIIdMdm4iIGEXFJiIiRlGxiYiIUVRsIiJiFBWbiIgY5a4vtvj4eKZOnXrTxyUkJJCZmcn333/PW2+9VQjJRETkVtz1xXarZs+ejcfjsTuGiIj8lyL5PbbExES++uor3G43p0+fpkePHmzatImDBw8yaNAgfvvtN9avX09WVhalSpVi6tSprFixgp07dzJp0iSio6MJDw/npZdeuu74//znP4mNjaVMmTL4+PjQsGFDABYtWsSaNWtwOBy0b9+eHj16MHjwYCzL4tdff+Xy5cuMGzeOnTt3cvr0ad566y169uzJzz//zGuvvUZKSgqPP/44AwYMuJOXS0RE/kORvWO7dOkSc+fOpXfv3sTHxzNt2jRiYmJYuXIl58+fJy4ujqVLl5KVlcXevXt56aWXuHLlCoMHDyYzM/OGpQYwZswYJk2axPz586latSoAhw4d4vPPP2fp0qUsXbqUjRs3cuTIEQD+53/+h4ULFzJgwAAmTJhA586dKV++PJMnTwYgPT2dGTNmsGTJEhYvXlz4F0dERG6oSN6xwf8v3VSqVCnCwsJwOByUKVOGzMxMnE4nb7/9NiVLluS3334jKysLgD59+tC1a1cSExPzHPvUqVOEhoYC0LhxY44dO0ZSUhInT54kKioKgAsXLnDs2DEAmjVrBkCjRo2IjY3NNV7NmjVzlsny8yuyl1T+j8vlsjvCLXG73cpuE2/OfzdmL7L/Cjscjutuz8zMZOPGjaxYsYIrV67QqVMnLMsiIyOD2NhYYmJiGDlyJEuWLLnhmozly5fn8OHDhIWFsXfvXsqUKUP16tWpUaMG8+bNw+FwEBcXR61atVi3bh379u2jSZMm7Ny5k5o1a+bku/ozthtllaLJW9e7dLlcym4Tb85vUvaCllyRLbYb8fPzo0SJEnTq1Al/f3/Kly9PcnIyEydO5LHHHqNr164kJyczadIkhgwZct0xJkyYQHR0NAEBAQQEBFCmTBnq1KnDH/7wB7p3705GRgbh4eFUqFABgK+//ppNmzbh8XgYM2YMAE2aNKFPnz7069fvjp27iIjkz2FZlmV3iKJs8ODBtG/fnpYtW962MV0uF+0WHLlt40nBefMiyCa98vY23pzfpOwFPRevu2MrqJMnTxIdHZ1re9OmTXnjjTdsSCQiIneCscVWuXJlFi1a9LvHGTt27G1IIyIid0qR/bi/iIjIrVCxiYiIUVRsIiJiFBWbiIgYxdgPjxRlHo/Hqz927s0uXUknoEQxu2OISCHSHZsNMjIy7I5wy7x1aZ6rjh3V9wdFTKdiExERo6jYRETEKCo2ERExiopNRESMomITERGjqNhERMQoKjYRETGKik1ERIyiYhMREaOo2ERExCgqNhERMYqKTUREjKJiExERo6jYRETEKCo2ERExiopNRESMomITERGjqNhERMQoKjYRETGKik1ERIyiYrOBv7+/3RFuWd26dW9qf3dmdiElERG5Pj+7A9yNfHx8qDZ4rd0x7oijYzvYHUFE7jK6YxMREaOo2ERExCgqNhERMYqKTUREjKJiExERo6jYRETEKLet2CIjIzl8+PDtGi5PTzzxBOnp6Td1zNSpU4mPjy/Qvhs2bODUqVO3Ek1ERGymO7brWLhwIWlpaXbHEBGRW5DvF7TdbjdDhgzh5MmTZGZmMnToUBISEjh+/DjZ2dn06tWL9u3b5+y/Y8cOxo0bh5+fH6VLl2bixIkADBs2jNTUVM6dO0fnzp2JiIggMjKS2rVrc/DgQUqWLEmTJk345ptvuHjxIp988gmbNm1i06ZNpKWlce7cOfr168fTTz+dM9evv/7Ku+++S3p6OsWKFWPUqFFUqlTphueyceNGvvjiC9xuN8OHD+fixYssX76cjz/+GIBu3boRERGBy+UiOjqaCRMm8MYbbxAUFETLli1p2bIlo0ePBiAoKIjY2FhKlSrFpEmT2L59O5ZlERUVRbt27W7tT0NERH63fItt2bJlVKlShcmTJ5OUlMTGjRsJDg5mwoQJpKWl0alTJ5o1a5az/8aNG3nyySd59dVX+fLLL7l48SLnzp2jQ4cOPPXUU5w6dYrIyEgiIiIACA8PZ/jw4bz66qsUL16c+fPnEx0dzfbt2wG4fPky8+fPJyUlhc6dO9O6deucucaNG0dkZCStWrXiH//4BxMnTmTSpEk3PJcqVaoQExPDwYMHGTRoEImJiYwePZoLFy5w+vRpgoODee6551ixYgUjR47E6XRy+vRpVq1ahb+/P126dCE2NpYaNWqwYsUK5s2bR+PGjTlx4gTLli0jPT2dLl260Lx5c0qXLn3LfyimcblcdkfI4Xa7i1Sem6Hs9vHm/Hdj9nyL7ciRI7Rs2RKAWrVqER8fzyOPPAJAYGAgYWFhHD9+PGf/vn37MmvWLHr27EmFChUIDw+nXLlyLFiwgPXr1xMYGEhWVlbO/vXr1wegdOnS1KhRI+fXV3+G1rRpU3x8fChXrhylS5cmJSUl59ikpCRmz57NvHnzsCwLp9OZ57k0bdoUgJo1a3L69GkcDgfPPfcca9as4cSJE7z44ou5jqlatWrO2o6HDx/m/fffByAzM5PQ0FCSkpLYt28fkZGRAGRlZXHy5EkV23+42fUlC5PL5SpSeW6GstvHm/OblL2gJZdvsYWFhbF3717atGnD8ePHWbt2Lf7+/jz55JOkpaWRlJRE1apVc/ZfvXo1HTt2JDo6mtmzZ7N8+XJSU1Np2LAhERERfPfdd2zevLnAJ7Zv3z4Azpw5Q1paGiEhITmPVa9enVdeeYXGjRtz+PDhnLu8G9mzZw/PPvssBw4coHLlygC88MILDBw4kCtXrvDOO+8A4HA4sCwL+Pe6jleFhoYybtw4KleuzI4dOzh9+jROp5OHH36YUaNG4fF4mDFjxjXXQ0RE7qx8i61bt24MHTqUl19+mezsbObNm8eSJUvo3r076enp9O/f/5qyadCgAYMHD6ZkyZI4nU5iYmL45ZdfGDlyJKtXryYoKAhfX18yMjIKFPDMmTP07NmT1NRURowYga+vb85j0dHRjBw5kvT0dNxuN8OGDctzrBMnTtCjRw8yMjKIiYkBoEKFCgQEBNCwYUP8/P59ORo1asSgQYMYNWrUNcePHDmS6OhosrP/vWL9Bx98QLVq1di2bRsRERFcvnyZNm3aEBgYWKBzExGR289hXb01KYISExM5cuQIAwcOLNR5/vjHPzJ06FDuu+++Qp3nKpfLRbsFR+7IXHYraqv7m/S2jDfx5uzg3flNyl7QczHuv63p378/Fy5cuGZbYGAgM2fOzLWv2+0mIiKCFi1a3LFSExGRwlWki61Tp043fcy0adMKvG/x4sVJTEy86TlERKTo0he0RUTEKCo2ERExiopNRESMomITERGjFOkPj5jK4/EUuY/BFxZ3ZjbFnb757ygicpvojs0GBf1yelF0s+u2qdRE5E5TsYmIiFFUbCIiYhQVm4iIGEXFJiIiRlGxiYiIUVRsIiJiFBWbiIgYRcUmIiJGUbGJiIhRVGwiImIUFZuIiBhFxSYiIkZRsYmIiFFUbCIiYhQVm4iIGEXFJiIiRlGxiYiIUVRsIiJiFBWbiIgYRcUmIiJGUbHZwM/Pz+4IIiLGUrHZQMUmIlJ4VGwiImIUFZuIiBhFxSYiIkZRsYmIiFFUbCIiYpTfXWz9+/e/HTluyOVyMW3atJs+7sCBA2zfvr0QEv2/5s2bA/DBBx9w8uRJzp8/z+rVqwt1ThERydvvLrZbKZ2bUbdu3Vsqz/Xr13Po0KFCSJTbsGHDqFy5MgcOHODLL7+8I3OKiMj15fuFqsTERL766ivcbjenT5+mR48ebNq0iYMHDzJo0CBGjBjB1q1biYyMpE6dOhw8eJC0tDSmTJlClSpVrjtmSkoKAwcOJCMjg9DQUL777js2bNjAunXrWLJkSc5+U6ZM4eDBgyxbtozJkyfz1FNP0bhxY3766SdCQkKYOnUqvr6+ucY/deoUn376KU6nk/r165OamspHH31EsWLFCAoKIjY2ltKlS1832/r165k7dy5+fn5UqVKF8ePHM336dI4cOcLZs2e5ePEiw4cPp0mTJjnHREZGMnLkSGbNmsX+/ftJSEiga9eu+V58ERG5/Qp0x3bp0iXmzp1L7969iY+PZ9q0acTExJCYmHjNfuHh4cTFxdG8eXPWrl17w/FmzZpF69atWbx4MW3btiU7OxuAo0ePMmfOHBYtWkRoaCjffPPNNccdP36cN998k4SEBFJSUti7d+91x69QoQIdO3YkKiqKBg0a8O677zJt2jQWL15M06ZNmTlz5g2zrVmzhqioKOLj43n00UdJS0sDoHjx4ixcuJAJEyYQExNz3WP79u1Ls2bNVGoiIjYq0BIYdevWBaBUqVKEhYXhcDgoU6YM6enp1+xXr149ACpWrMiZM2duON7hw4fp2LEjwDV3PiEhIURHRxMQEMCRI0do2LDhNccFBwdTqVIlACpVqpRr/us5d+4cgYGBVKhQAYCmTZvy4Ycf3nD/IUOGMHv2bOLj46levTpt2rQBoFmzZgDUrFkzz3MrKJfL9bvHsIPb7fba7ODd+ZXdPt6c/27MXqBiczgcNz1wXmrVqsWuXbuoW7cuu3fvBiA1NZWPP/6Yv//97wD06tULy7JuOYfD4cDj8RAcHExaWhrJycncc889bNu2jWrVqt3wuISEBAYMGEBISAjvvfceGzZsAGDfvn08//zzJCUl5ZTkf/Px8cHj8RQo39UXC97G5XJ5bXbw7vzKbh9vzm9S9oKWnC2LFvbu3ZtBgwbxxRdfcM899+Dn50dgYCCNGzemY8eOlCxZktKlS5OcnEzVqlVvaY7777+f8ePHExYWxujRoxkwYEDOneaYMWNueFx4eDi9evUiKCiIgIAAHnvsMRYvXozL5aJnz55cuXKFUaNGXffYe++9l6SkJOLi4oiKirql3CIi8vs4rP++LboDNm/eTHBwMOHh4Xz77bfMmjWLhQsX3ukYBTZ16lTKlStH9+7db8t4Jr2C8jbenF/Z7ePN+U3KXtBzKdQ7tv79+3PhwoVrtgUGBjJw4ECGDh2Kr68vHo+HYcOG3dL4J0+eJDo6Otf2pk2b8sYbb9zwuIyMDF599dVc20NDQ2/4wRAREfEOhVpseX3HLSEh4XePX7lyZRYtWnTTx/n7+9/UcQMGDLjpOURExB5aUktERIyiYhMREaOo2ERExCgqNhERMYqKzQZZWVl2RxARMZaKzQYqNhGRwqNiExERo6jYRETEKCo2ERExiopNRESMomITERGjqNhERMQoKjYRETGKik1ERIyiYhMREaOo2ERExCgqNhERMYqKTUREjKJiExERo6jYRETEKCo2ERExiopNRESMomITERGjqNhERMQoKjYRETGKik1ERIyiYrOBn5+f3RFERIylYrOBik1EpPCo2ERExCgqNhERMYqKTUREjKJiExERo6jYRETEKCo2ERExitcXW2JiIhMnTrytY54/f57Vq1cDMGfOHPbs2XNbxxcRkcLj9cVWGA4cOMCXX34JQJ8+fQgPD7c5kYiIFJQR3xTevXs3PXv2JC0tjQEDBuB0Ovnoo48oVqwYQUFBxMbGUrp0acaOHcuOHTsAeOaZZ+jZsyfr169n7ty5+Pn5UaVKFcaPH8+sWbPYv38/CQkJ7Nq1i/bt23PmzBk2b96M2+3m2LFj9O7dm06dOrFnzx7ef/99AgICCAkJoVixYowdO9bmKyIicvcyothKlCjBnDlzSElJoXPnzgDEx8dToUIFFixYwMyZM3nooYc4ceIEy5cvJysri4iICJo1a8aaNWuIioqiQ4cOfPbZZ6SlpdG3b1+WLVtG165d2bVrV848aWlp/OUvf+Ho0aP07duXTp06MWLECMaPH0/NmjWZPHkyp06dKlBml8tVKNeisLndbq/NDt6dX9nt483578bsRhTbgw8+iMPhICQkhBIlSgBQoUIFAJo2bcqHH35ISEgITZo0weFw4HQ6eeCBBzh8+DBDhgxh9uzZxMfHU716ddq0aXPDeerUqQNApUqVyMjIACA5OZmaNWvm5Pj8888LlLlu3bq3fL52crlcXpsdvDu/stvHm/OblL2gJWfEz9j27t0LwOnTp0lPT+fKlSskJycDsG3bNqpVq0ZYWFjO25CZmZns2rWL++67j4SEBAYMGMDixYsB2LBhAz4+Png8nlzzOByOXNsqVqzIoUOHAPjhhx8K5fxERKTgjLhjc7vd9OjRg8uXLzNq1Cgsy2LAgAE4HA7KlCnDmDFjKFu2LNu2baNr165kZmbStm1b6tevz6lTp+jVqxdBQUEEBATw2GOPkZGRQVJSEnFxcfnOPWLECIYOHUrJkiVxOp05d4oiImIPry+2Tp060alTp1zbH3nkkVzboqOjc2174okneOKJJ3Jt/+KLL244Z7FixXI+Nbl3715mzZpF2bJlmTx5Mk6n82bii4jIbeb1xWa3kJAQXnnlFUqWLEmpUqX0iUgREZup2H6ntm3b0rZtW7tjiIjI/zHiwyMiIiJXqdhERMQoKjYRETGKik1ERIyiYrNBVlaW3RFERIylYrOBik1EpPCo2ERExCgqNhERMYqKTUREjKJiExERo6jYRETEKCo2ERExiopNRESMomITERGjqNhERMQoDsuyLLtD3G12795NsWLF7I4hIuJV0tPTadiwYb77qdhERMQoeitSRESMomITERGjqNhERMQoKjYRETGKik1ERIziZ3cAU3k8HkaOHMmBAwfw9/dn9OjR3HfffTmPL1++nGXLluHn58ef/vQnHn/8cRvT5pZffoCUlBS6devG6tWri9TXF/LLHhcXx9q1awFo1aoV/fv3tytqLvllX7JkCYmJiTgcDvr16+eVzxuPx0OfPn1o3bo13bt3tylpbvllHz16NDt37iQgIACAGTNmUKpUKbviXiO/7Js3b2b69OkA1KtXjxEjRuBwOOyKm0te+V0uF7GxsTn77t69m+nTp9OyZcsbD2hJofjb3/5mRUdHW5ZlWbt27bL69u2b81hycrL1zDPPWOnp6dbFixdzfl2U5JXfsizr66+/tp5//nmrUaNGltvttiPiDeWV/dixY1bHjh2trKwsKzs72+ratavlcrnsippLXtnPnj1rtW/f3srIyLBSU1Otli1bWh6Px66o15Xf88ayLGvSpEnWiy++aC1duvROx8tTftm7detmnT171o5o+core2pqqtWhQ4ec7HPmzCly51GQ541lWdbnn39uvf322/mOp7ciC8mOHTto0aIFAA0bNuTHH3/MeWzPnj00atQIf39/SpUqxb333sv+/fvtinpdeeUH8PHxYf78+QQFBdkRL095Za9YsSLz5s3D19cXHx8fsrKyitTdZl7Zy5Yty1//+lecTidnzpyhdOnSRepVN+T/vFm3bh0OhyPvV9s2ySu7x+Ph559/5r333qNbt26sXLnSrpjXlVf2Xbt2UatWLcaNG0dERATlypWjbNmydkW9rvyeNwCXL19m6tSpDBs2LN/xVGyFJC0tjcDAwJzf+/r6kpWVlfPYf76FERAQQFpa2h3PmJe88gM0b96c4OBgO6LlK6/sTqeTsmXLYlkW48aNo169eoSGhtoVNZf8rrufnx+LFy+ma9euPP3003ZEzFNe+ZOSklizZg1vvvmmXfHylFf2y5cv8/LLLzNhwgTmzZvH0qVLi9SL0byynzt3ju+//56BAwcyd+5cFixYwE8//WRX1OvK73kPsHLlStq2bVugUlaxFZLAwEAuXbqU83uPx4Ofn991H7t06VKRea/+qrzyF3X5ZU9PT2fgwIFcunSJESNG2BHxhgpy3V9++WW2bNnC9u3b+e677+50xDzllf+zzz7j1KlT9OzZk08//ZS4uDi+/vpru6Lmklf2EiVK0KNHD0qUKEFgYCDNmjUrUsWWV/agoCAaNGhA+fLlCQgIoEmTJrhcLruiXldBnverV6+mc+fOBRpPxVZIGjdunPOXdvfu3dSqVSvnsfDwcHbs2EF6ejqpqakcPnz4mseLgrzyF3V5Zbcsi9dff53atWsTExODr6+vXTGvK6/sR44coX///liWhdPpxN/fHx+fovVXOK/8gwYNYsWKFSxatIiOHTsSFRVVpN6SzCv70aNHiYiIIDs7m8zMTHbu3En9+vXtippLXtnvv/9+kpKSSElJISsrix9++IEaNWrYFfW68vv3JjU1lYyMDCpVqlSg8bzjJbgXevLJJ9m6dSvdunXDsixiY2OZP38+9957L61btyYyMpKIiAgsy+Ktt94qUj/ngfzzF2V5Zfd4PGzbto2MjAy2bNkCwNtvv02jRo1sTv1v+V33OnXq0LVrVxwOBy1atOChhx6yO/I1TH3etG7dmmeffZYuXbrgdDp5/vnnqVmzpt2Rc+SX/Z133uG1114DoG3btkXuhWp++X/66SeqVKlS4PG0CLKIiBilaL2PISIi8jup2ERExCgqNhERMYqKTUREjKJiExERo6jYRETEKCo2ERExiopNRESM8r8A6bHHA7ECkQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "pd.Series(reg_tree.feature_importances_, index=xgb_features).sort_values().plot.barh();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-23T02:10:09.761192Z",
     "start_time": "2018-10-23T02:10:09.638515Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.40.1 (20161225.0304)\n",
       " -->\n",
       "<!-- Title: Tree Pages: 1 -->\n",
       "<svg width=\"1116pt\" height=\"477pt\"\n",
       " viewBox=\"0.00 0.00 1116.00 477.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 473)\">\n",
       "<title>Tree</title>\n",
       "<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-4,4 -4,-473 1112,-473 1112,4 -4,4\"/>\n",
       "<!-- 0 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>0</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.776471\" stroke=\"#000000\" d=\"M547.5,-469C547.5,-469 426.5,-469 426.5,-469 420.5,-469 414.5,-463 414.5,-457 414.5,-457 414.5,-413 414.5,-413 414.5,-407 420.5,-401 426.5,-401 426.5,-401 547.5,-401 547.5,-401 553.5,-401 559.5,-407 559.5,-413 559.5,-413 559.5,-457 559.5,-457 559.5,-463 553.5,-469 547.5,-469\"/>\n",
       "<text text-anchor=\"start\" x=\"422.5\" y=\"-453.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">learning_rate ≤ 0.055</text>\n",
       "<text text-anchor=\"start\" x=\"449\" y=\"-438.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.001</text>\n",
       "<text text-anchor=\"start\" x=\"442\" y=\"-423.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 216</text>\n",
       "<text text-anchor=\"start\" x=\"446\" y=\"-408.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.652</text>\n",
       "</g>\n",
       "<!-- 1 -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>1</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.431373\" stroke=\"#000000\" d=\"M453.5,-365C453.5,-365 358.5,-365 358.5,-365 352.5,-365 346.5,-359 346.5,-353 346.5,-353 346.5,-309 346.5,-309 346.5,-303 352.5,-297 358.5,-297 358.5,-297 453.5,-297 453.5,-297 459.5,-297 465.5,-303 465.5,-309 465.5,-309 465.5,-353 465.5,-353 465.5,-359 459.5,-365 453.5,-365\"/>\n",
       "<text text-anchor=\"start\" x=\"354.5\" y=\"-349.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">max_depth ≤ 6.0</text>\n",
       "<text text-anchor=\"start\" x=\"368\" y=\"-334.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.001</text>\n",
       "<text text-anchor=\"start\" x=\"365\" y=\"-319.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 72</text>\n",
       "<text text-anchor=\"start\" x=\"365\" y=\"-304.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.612</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;1 -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>0&#45;&gt;1</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M460.4776,-400.9465C453.5546,-392.0578 446.0106,-382.3716 438.804,-373.1188\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"441.488,-370.8688 432.582,-365.13 435.9654,-375.1701 441.488,-370.8688\"/>\n",
       "<text text-anchor=\"middle\" x=\"429.4977\" y=\"-386.239\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">True</text>\n",
       "</g>\n",
       "<!-- 12 -->\n",
       "<g id=\"node13\" class=\"node\">\n",
       "<title>12</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.945098\" stroke=\"#000000\" d=\"M648.5,-365C648.5,-365 553.5,-365 553.5,-365 547.5,-365 541.5,-359 541.5,-353 541.5,-353 541.5,-309 541.5,-309 541.5,-303 547.5,-297 553.5,-297 553.5,-297 648.5,-297 648.5,-297 654.5,-297 660.5,-303 660.5,-309 660.5,-309 660.5,-353 660.5,-353 660.5,-359 654.5,-365 648.5,-365\"/>\n",
       "<text text-anchor=\"start\" x=\"549.5\" y=\"-349.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">max_depth ≤ 4.0</text>\n",
       "<text text-anchor=\"start\" x=\"570.5\" y=\"-334.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"556\" y=\"-319.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 144</text>\n",
       "<text text-anchor=\"start\" x=\"560\" y=\"-304.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.672</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;12 -->\n",
       "<g id=\"edge12\" class=\"edge\">\n",
       "<title>0&#45;&gt;12</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M524.3279,-400.9465C534.465,-391.6986 545.5483,-381.5876 556.0583,-371.9994\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"558.5595,-374.4553 563.5883,-365.13 553.8417,-369.284 558.5595,-374.4553\"/>\n",
       "<text text-anchor=\"middle\" x=\"562.4427\" y=\"-386.4037\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">False</text>\n",
       "</g>\n",
       "<!-- 2 -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>2</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.117647\" stroke=\"#000000\" d=\"M232,-261C232,-261 98,-261 98,-261 92,-261 86,-255 86,-249 86,-249 86,-205 86,-205 86,-199 92,-193 98,-193 98,-193 232,-193 232,-193 238,-193 244,-199 244,-205 244,-205 244,-249 244,-249 244,-255 238,-261 232,-261\"/>\n",
       "<text text-anchor=\"start\" x=\"94\" y=\"-245.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">colsample_bytree ≤ 0.9</text>\n",
       "<text text-anchor=\"start\" x=\"134.5\" y=\"-230.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"124\" y=\"-215.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 24</text>\n",
       "<text text-anchor=\"start\" x=\"124\" y=\"-200.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.575</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;2 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>1&#45;&gt;2</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M346.427,-305.2921C318.3837,-293.1905 284.366,-278.5107 253.3992,-265.1474\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"254.4651,-261.7954 243.8968,-261.0467 251.6916,-268.2225 254.4651,-261.7954\"/>\n",
       "</g>\n",
       "<!-- 5 -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>5</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.588235\" stroke=\"#000000\" d=\"M473,-261C473,-261 339,-261 339,-261 333,-261 327,-255 327,-249 327,-249 327,-205 327,-205 327,-199 333,-193 339,-193 339,-193 473,-193 473,-193 479,-193 485,-199 485,-205 485,-205 485,-249 485,-249 485,-255 479,-261 473,-261\"/>\n",
       "<text text-anchor=\"start\" x=\"335\" y=\"-245.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">colsample_bytree ≤ 0.9</text>\n",
       "<text text-anchor=\"start\" x=\"368\" y=\"-230.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.001</text>\n",
       "<text text-anchor=\"start\" x=\"365\" y=\"-215.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 48</text>\n",
       "<text text-anchor=\"start\" x=\"368.5\" y=\"-200.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.63</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;5 -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>1&#45;&gt;5</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M406,-296.9465C406,-288.776 406,-279.9318 406,-271.3697\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"409.5001,-271.13 406,-261.13 402.5001,-271.13 409.5001,-271.13\"/>\n",
       "</g>\n",
       "<!-- 3 -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>3</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.235294\" stroke=\"#000000\" d=\"M86,-149.5C86,-149.5 12,-149.5 12,-149.5 6,-149.5 0,-143.5 0,-137.5 0,-137.5 0,-108.5 0,-108.5 0,-102.5 6,-96.5 12,-96.5 12,-96.5 86,-96.5 86,-96.5 92,-96.5 98,-102.5 98,-108.5 98,-108.5 98,-137.5 98,-137.5 98,-143.5 92,-149.5 86,-149.5\"/>\n",
       "<text text-anchor=\"start\" x=\"18.5\" y=\"-134.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"8\" y=\"-119.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 12</text>\n",
       "<text text-anchor=\"start\" x=\"8\" y=\"-104.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.589</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;3 -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>2&#45;&gt;3</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M127.0173,-192.9465C113.8714,-181.1606 99.1618,-167.9726 86.0887,-156.2519\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"88.3869,-153.6116 78.6047,-149.5422 83.714,-158.8236 88.3869,-153.6116\"/>\n",
       "</g>\n",
       "<!-- 4 -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>4</title>\n",
       "<path fill=\"transparent\" stroke=\"#000000\" d=\"M202,-149.5C202,-149.5 128,-149.5 128,-149.5 122,-149.5 116,-143.5 116,-137.5 116,-137.5 116,-108.5 116,-108.5 116,-102.5 122,-96.5 128,-96.5 128,-96.5 202,-96.5 202,-96.5 208,-96.5 214,-102.5 214,-108.5 214,-108.5 214,-137.5 214,-137.5 214,-143.5 208,-149.5 202,-149.5\"/>\n",
       "<text text-anchor=\"start\" x=\"134.5\" y=\"-134.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"124\" y=\"-119.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 12</text>\n",
       "<text text-anchor=\"start\" x=\"124\" y=\"-104.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.561</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;4 -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>2&#45;&gt;4</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M165,-192.9465C165,-182.2621 165,-170.4254 165,-159.5742\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"168.5001,-159.5421 165,-149.5422 161.5001,-159.5422 168.5001,-159.5421\"/>\n",
       "</g>\n",
       "<!-- 6 -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>6</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.796078\" stroke=\"#000000\" d=\"M346,-157C346,-157 244,-157 244,-157 238,-157 232,-151 232,-145 232,-145 232,-101 232,-101 232,-95 238,-89 244,-89 244,-89 346,-89 346,-89 352,-89 358,-95 358,-101 358,-101 358,-145 358,-145 358,-151 352,-157 346,-157\"/>\n",
       "<text text-anchor=\"start\" x=\"240\" y=\"-141.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">max_depth ≤ 10.0</text>\n",
       "<text text-anchor=\"start\" x=\"264.5\" y=\"-126.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"254\" y=\"-111.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 24</text>\n",
       "<text text-anchor=\"start\" x=\"254\" y=\"-96.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.655</text>\n",
       "</g>\n",
       "<!-- 5&#45;&gt;6 -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>5&#45;&gt;6</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M369.6544,-192.9465C359.7841,-183.6986 348.9925,-173.5876 338.759,-163.9994\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"341.1177,-161.4131 331.4272,-157.13 336.3316,-166.5214 341.1177,-161.4131\"/>\n",
       "</g>\n",
       "<!-- 9 -->\n",
       "<g id=\"node10\" class=\"node\">\n",
       "<title>9</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.384314\" stroke=\"#000000\" d=\"M490,-157C490,-157 388,-157 388,-157 382,-157 376,-151 376,-145 376,-145 376,-101 376,-101 376,-95 382,-89 388,-89 388,-89 490,-89 490,-89 496,-89 502,-95 502,-101 502,-101 502,-145 502,-145 502,-151 496,-157 490,-157\"/>\n",
       "<text text-anchor=\"start\" x=\"384\" y=\"-141.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">max_depth ≤ 10.0</text>\n",
       "<text text-anchor=\"start\" x=\"408.5\" y=\"-126.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"398\" y=\"-111.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 24</text>\n",
       "<text text-anchor=\"start\" x=\"398\" y=\"-96.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.606</text>\n",
       "</g>\n",
       "<!-- 5&#45;&gt;9 -->\n",
       "<g id=\"edge9\" class=\"edge\">\n",
       "<title>5&#45;&gt;9</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M416.8054,-192.9465C419.455,-184.5965 422.3278,-175.5427 425.1001,-166.8056\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"428.4818,-167.7203 428.1703,-157.13 421.8097,-165.6031 428.4818,-167.7203\"/>\n",
       "</g>\n",
       "<!-- 7 -->\n",
       "<g id=\"node8\" class=\"node\">\n",
       "<title>7</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.737255\" stroke=\"#000000\" d=\"M223,-53C223,-53 149,-53 149,-53 143,-53 137,-47 137,-41 137,-41 137,-12 137,-12 137,-6 143,0 149,0 149,0 223,0 223,0 229,0 235,-6 235,-12 235,-12 235,-41 235,-41 235,-47 229,-53 223,-53\"/>\n",
       "<text text-anchor=\"start\" x=\"155.5\" y=\"-37.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"145\" y=\"-22.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 12</text>\n",
       "<text text-anchor=\"start\" x=\"145\" y=\"-7.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.648</text>\n",
       "</g>\n",
       "<!-- 6&#45;&gt;7 -->\n",
       "<g id=\"edge7\" class=\"edge\">\n",
       "<title>6&#45;&gt;7</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M256.5707,-88.9777C245.9173,-79.546 234.3641,-69.3178 223.7499,-59.9208\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"226.0207,-57.2566 216.2133,-53.2485 221.3806,-62.4978 226.0207,-57.2566\"/>\n",
       "</g>\n",
       "<!-- 8 -->\n",
       "<g id=\"node9\" class=\"node\">\n",
       "<title>8</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.858824\" stroke=\"#000000\" d=\"M339,-53C339,-53 265,-53 265,-53 259,-53 253,-47 253,-41 253,-41 253,-12 253,-12 253,-6 259,0 265,0 265,0 339,0 339,0 345,0 351,-6 351,-12 351,-12 351,-41 351,-41 351,-47 345,-53 339,-53\"/>\n",
       "<text text-anchor=\"start\" x=\"271.5\" y=\"-37.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"261\" y=\"-22.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 12</text>\n",
       "<text text-anchor=\"start\" x=\"261\" y=\"-7.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.661</text>\n",
       "</g>\n",
       "<!-- 6&#45;&gt;8 -->\n",
       "<g id=\"edge8\" class=\"edge\">\n",
       "<title>6&#45;&gt;8</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M297.4679,-88.9777C298.0724,-80.6449 298.722,-71.6903 299.3352,-63.2364\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"302.8269,-63.4755 300.0597,-53.2485 295.8453,-62.969 302.8269,-63.4755\"/>\n",
       "</g>\n",
       "<!-- 10 -->\n",
       "<g id=\"node11\" class=\"node\">\n",
       "<title>10</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.333333\" stroke=\"#000000\" d=\"M462,-53C462,-53 388,-53 388,-53 382,-53 376,-47 376,-41 376,-41 376,-12 376,-12 376,-6 382,0 388,0 388,0 462,0 462,0 468,0 474,-6 474,-12 474,-12 474,-41 474,-41 474,-47 468,-53 462,-53\"/>\n",
       "<text text-anchor=\"start\" x=\"394.5\" y=\"-37.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"384\" y=\"-22.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 12</text>\n",
       "<text text-anchor=\"start\" x=\"391.5\" y=\"-7.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.6</text>\n",
       "</g>\n",
       "<!-- 9&#45;&gt;10 -->\n",
       "<g id=\"edge10\" class=\"edge\">\n",
       "<title>9&#45;&gt;10</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M434.0641,-88.9777C432.8552,-80.6449 431.5561,-71.6903 430.3296,-63.2364\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"433.7802,-62.6423 428.8806,-53.2485 426.8527,-63.6474 433.7802,-62.6423\"/>\n",
       "</g>\n",
       "<!-- 11 -->\n",
       "<g id=\"node12\" class=\"node\">\n",
       "<title>11</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.435294\" stroke=\"#000000\" d=\"M578,-53C578,-53 504,-53 504,-53 498,-53 492,-47 492,-41 492,-41 492,-12 492,-12 492,-6 498,0 504,0 504,0 578,0 578,0 584,0 590,-6 590,-12 590,-12 590,-41 590,-41 590,-47 584,-53 578,-53\"/>\n",
       "<text text-anchor=\"start\" x=\"510.5\" y=\"-37.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"500\" y=\"-22.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 12</text>\n",
       "<text text-anchor=\"start\" x=\"500\" y=\"-7.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.612</text>\n",
       "</g>\n",
       "<!-- 9&#45;&gt;11 -->\n",
       "<g id=\"edge11\" class=\"edge\">\n",
       "<title>9&#45;&gt;11</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M474.9614,-88.9777C484.8338,-79.6376 495.532,-69.5163 505.3849,-60.1947\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"507.8682,-62.6635 512.727,-53.2485 503.0574,-57.5785 507.8682,-62.6635\"/>\n",
       "</g>\n",
       "<!-- 13 -->\n",
       "<g id=\"node14\" class=\"node\">\n",
       "<title>13</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.898039\" stroke=\"#000000\" d=\"M654,-261C654,-261 548,-261 548,-261 542,-261 536,-255 536,-249 536,-249 536,-205 536,-205 536,-199 542,-193 548,-193 548,-193 654,-193 654,-193 660,-193 666,-199 666,-205 666,-205 666,-249 666,-249 666,-255 660,-261 654,-261\"/>\n",
       "<text text-anchor=\"start\" x=\"544\" y=\"-245.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">learning_rate ≤ 0.2</text>\n",
       "<text text-anchor=\"start\" x=\"570.5\" y=\"-230.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"560\" y=\"-215.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 24</text>\n",
       "<text text-anchor=\"start\" x=\"560\" y=\"-200.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.666</text>\n",
       "</g>\n",
       "<!-- 12&#45;&gt;13 -->\n",
       "<g id=\"edge13\" class=\"edge\">\n",
       "<title>12&#45;&gt;13</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M601,-296.9465C601,-288.776 601,-279.9318 601,-271.3697\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"604.5001,-271.13 601,-261.13 597.5001,-271.13 604.5001,-271.13\"/>\n",
       "</g>\n",
       "<!-- 16 -->\n",
       "<g id=\"node17\" class=\"node\">\n",
       "<title>16</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.956863\" stroke=\"#000000\" d=\"M865,-261C865,-261 759,-261 759,-261 753,-261 747,-255 747,-249 747,-249 747,-205 747,-205 747,-199 753,-193 759,-193 759,-193 865,-193 865,-193 871,-193 877,-199 877,-205 877,-205 877,-249 877,-249 877,-255 871,-261 865,-261\"/>\n",
       "<text text-anchor=\"start\" x=\"755\" y=\"-245.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">learning_rate ≤ 0.2</text>\n",
       "<text text-anchor=\"start\" x=\"781.5\" y=\"-230.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"767\" y=\"-215.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 120</text>\n",
       "<text text-anchor=\"start\" x=\"771\" y=\"-200.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.673</text>\n",
       "</g>\n",
       "<!-- 12&#45;&gt;16 -->\n",
       "<g id=\"edge16\" class=\"edge\">\n",
       "<title>12&#45;&gt;16</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M660.6181,-301.6148C684.6046,-289.792 712.521,-276.0323 737.801,-263.572\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"739.5609,-266.6067 746.9832,-259.0462 736.4661,-260.3279 739.5609,-266.6067\"/>\n",
       "</g>\n",
       "<!-- 14 -->\n",
       "<g id=\"node15\" class=\"node\">\n",
       "<title>14</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.831373\" stroke=\"#000000\" d=\"M606,-149.5C606,-149.5 532,-149.5 532,-149.5 526,-149.5 520,-143.5 520,-137.5 520,-137.5 520,-108.5 520,-108.5 520,-102.5 526,-96.5 532,-96.5 532,-96.5 606,-96.5 606,-96.5 612,-96.5 618,-102.5 618,-108.5 618,-108.5 618,-137.5 618,-137.5 618,-143.5 612,-149.5 606,-149.5\"/>\n",
       "<text text-anchor=\"start\" x=\"538.5\" y=\"-134.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"528\" y=\"-119.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 12</text>\n",
       "<text text-anchor=\"start\" x=\"528\" y=\"-104.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.658</text>\n",
       "</g>\n",
       "<!-- 13&#45;&gt;14 -->\n",
       "<g id=\"edge14\" class=\"edge\">\n",
       "<title>13&#45;&gt;14</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M590.522,-192.9465C587.2006,-182.1519 583.5173,-170.1813 580.1504,-159.2389\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"583.453,-158.0706 577.1668,-149.5422 576.7625,-160.1293 583.453,-158.0706\"/>\n",
       "</g>\n",
       "<!-- 15 -->\n",
       "<g id=\"node16\" class=\"node\">\n",
       "<title>15</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.964706\" stroke=\"#000000\" d=\"M722,-149.5C722,-149.5 648,-149.5 648,-149.5 642,-149.5 636,-143.5 636,-137.5 636,-137.5 636,-108.5 636,-108.5 636,-102.5 642,-96.5 648,-96.5 648,-96.5 722,-96.5 722,-96.5 728,-96.5 734,-102.5 734,-108.5 734,-108.5 734,-137.5 734,-137.5 734,-143.5 728,-149.5 722,-149.5\"/>\n",
       "<text text-anchor=\"start\" x=\"654.5\" y=\"-134.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"644\" y=\"-119.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 12</text>\n",
       "<text text-anchor=\"start\" x=\"644\" y=\"-104.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.674</text>\n",
       "</g>\n",
       "<!-- 13&#45;&gt;15 -->\n",
       "<g id=\"edge15\" class=\"edge\">\n",
       "<title>13&#45;&gt;15</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M628.5047,-192.9465C637.6683,-181.6012 647.8811,-168.9567 657.0761,-157.5724\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"660.0015,-159.5208 663.5621,-149.5422 654.5559,-155.1224 660.0015,-159.5208\"/>\n",
       "</g>\n",
       "<!-- 17 -->\n",
       "<g id=\"node18\" class=\"node\">\n",
       "<title>17</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.980392\" stroke=\"#000000\" d=\"M859.5,-157C859.5,-157 764.5,-157 764.5,-157 758.5,-157 752.5,-151 752.5,-145 752.5,-145 752.5,-101 752.5,-101 752.5,-95 758.5,-89 764.5,-89 764.5,-89 859.5,-89 859.5,-89 865.5,-89 871.5,-95 871.5,-101 871.5,-101 871.5,-145 871.5,-145 871.5,-151 865.5,-157 859.5,-157\"/>\n",
       "<text text-anchor=\"start\" x=\"760.5\" y=\"-141.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">max_depth ≤ 8.0</text>\n",
       "<text text-anchor=\"start\" x=\"781.5\" y=\"-126.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"771\" y=\"-111.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 60</text>\n",
       "<text text-anchor=\"start\" x=\"771\" y=\"-96.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.676</text>\n",
       "</g>\n",
       "<!-- 16&#45;&gt;17 -->\n",
       "<g id=\"edge17\" class=\"edge\">\n",
       "<title>16&#45;&gt;17</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M812,-192.9465C812,-184.776 812,-175.9318 812,-167.3697\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"815.5001,-167.13 812,-157.13 808.5001,-167.13 815.5001,-167.13\"/>\n",
       "</g>\n",
       "<!-- 20 -->\n",
       "<g id=\"node21\" class=\"node\">\n",
       "<title>20</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.933333\" stroke=\"#000000\" d=\"M996.5,-157C996.5,-157 901.5,-157 901.5,-157 895.5,-157 889.5,-151 889.5,-145 889.5,-145 889.5,-101 889.5,-101 889.5,-95 895.5,-89 901.5,-89 901.5,-89 996.5,-89 996.5,-89 1002.5,-89 1008.5,-95 1008.5,-101 1008.5,-101 1008.5,-145 1008.5,-145 1008.5,-151 1002.5,-157 996.5,-157\"/>\n",
       "<text text-anchor=\"start\" x=\"897.5\" y=\"-141.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">max_depth ≤ 8.0</text>\n",
       "<text text-anchor=\"start\" x=\"918.5\" y=\"-126.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"908\" y=\"-111.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 60</text>\n",
       "<text text-anchor=\"start\" x=\"911.5\" y=\"-96.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.67</text>\n",
       "</g>\n",
       "<!-- 16&#45;&gt;20 -->\n",
       "<g id=\"edge20\" class=\"edge\">\n",
       "<title>16&#45;&gt;20</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M856.8589,-192.9465C869.2778,-183.519 882.8783,-173.1946 895.7261,-163.4415\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"898.1915,-165.9642 904.0403,-157.13 893.959,-160.3887 898.1915,-165.9642\"/>\n",
       "</g>\n",
       "<!-- 18 -->\n",
       "<g id=\"node19\" class=\"node\">\n",
       "<title>18</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.952941\" stroke=\"#000000\" d=\"M738,-53C738,-53 664,-53 664,-53 658,-53 652,-47 652,-41 652,-41 652,-12 652,-12 652,-6 658,0 664,0 664,0 738,0 738,0 744,0 750,-6 750,-12 750,-12 750,-41 750,-41 750,-47 744,-53 738,-53\"/>\n",
       "<text text-anchor=\"start\" x=\"670.5\" y=\"-37.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"660\" y=\"-22.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 24</text>\n",
       "<text text-anchor=\"start\" x=\"660\" y=\"-7.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.673</text>\n",
       "</g>\n",
       "<!-- 17&#45;&gt;18 -->\n",
       "<g id=\"edge18\" class=\"edge\">\n",
       "<title>17&#45;&gt;18</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M772.8656,-88.9777C762.0167,-79.546 750.2516,-69.3178 739.4426,-59.9208\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"741.6108,-57.1681 731.7677,-53.2485 737.0181,-62.4508 741.6108,-57.1681\"/>\n",
       "</g>\n",
       "<!-- 19 -->\n",
       "<g id=\"node20\" class=\"node\">\n",
       "<title>19</title>\n",
       "<path fill=\"#e58139\" stroke=\"#000000\" d=\"M854,-53C854,-53 780,-53 780,-53 774,-53 768,-47 768,-41 768,-41 768,-12 768,-12 768,-6 774,0 780,0 780,0 854,0 854,0 860,0 866,-6 866,-12 866,-12 866,-41 866,-41 866,-47 860,-53 854,-53\"/>\n",
       "<text text-anchor=\"start\" x=\"786.5\" y=\"-37.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"776\" y=\"-22.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 36</text>\n",
       "<text text-anchor=\"start\" x=\"776\" y=\"-7.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.678</text>\n",
       "</g>\n",
       "<!-- 17&#45;&gt;19 -->\n",
       "<g id=\"edge19\" class=\"edge\">\n",
       "<title>17&#45;&gt;19</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M813.7628,-88.9777C814.1946,-80.6449 814.6585,-71.6903 815.0966,-63.2364\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"818.5918,-63.4162 815.6141,-53.2485 811.6012,-63.0539 818.5918,-63.4162\"/>\n",
       "</g>\n",
       "<!-- 21 -->\n",
       "<g id=\"node22\" class=\"node\">\n",
       "<title>21</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.945098\" stroke=\"#000000\" d=\"M980,-53C980,-53 906,-53 906,-53 900,-53 894,-47 894,-41 894,-41 894,-12 894,-12 894,-6 900,0 906,0 906,0 980,0 980,0 986,0 992,-6 992,-12 992,-12 992,-41 992,-41 992,-47 986,-53 980,-53\"/>\n",
       "<text text-anchor=\"start\" x=\"912.5\" y=\"-37.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"902\" y=\"-22.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 24</text>\n",
       "<text text-anchor=\"start\" x=\"902\" y=\"-7.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.672</text>\n",
       "</g>\n",
       "<!-- 20&#45;&gt;21 -->\n",
       "<g id=\"edge21\" class=\"edge\">\n",
       "<title>20&#45;&gt;21</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M946.8846,-88.9777C946.3665,-80.6449 945.8098,-71.6903 945.2841,-63.2364\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"948.777,-63.012 944.6631,-53.2485 941.7905,-63.4464 948.777,-63.012\"/>\n",
       "</g>\n",
       "<!-- 22 -->\n",
       "<g id=\"node23\" class=\"node\">\n",
       "<title>22</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.921569\" stroke=\"#000000\" d=\"M1096,-53C1096,-53 1022,-53 1022,-53 1016,-53 1010,-47 1010,-41 1010,-41 1010,-12 1010,-12 1010,-6 1016,0 1022,0 1022,0 1096,0 1096,0 1102,0 1108,-6 1108,-12 1108,-12 1108,-41 1108,-41 1108,-47 1102,-53 1096,-53\"/>\n",
       "<text text-anchor=\"start\" x=\"1028.5\" y=\"-37.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"1018\" y=\"-22.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 36</text>\n",
       "<text text-anchor=\"start\" x=\"1018\" y=\"-7.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.669</text>\n",
       "</g>\n",
       "<!-- 20&#45;&gt;22 -->\n",
       "<g id=\"edge22\" class=\"edge\">\n",
       "<title>20&#45;&gt;22</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M987.7819,-88.9777C998.533,-79.546 1010.1922,-69.3178 1020.9038,-59.9208\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1023.3004,-62.4743 1028.5095,-53.2485 1018.684,-57.2122 1023.3004,-62.4743\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.files.Source at 0x7f0d84b62978>"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "out_file = 'results/xgb_tree.dot'\n",
    "dot_data = export_graphviz(reg_tree,\n",
    "                          out_file=out_file,\n",
    "                          feature_names=xgb_features,\n",
    "                          max_depth=4,\n",
    "                          filled=True,\n",
    "                          rounded=True,\n",
    "                          special_characters=True)\n",
    "if out_file is not None:\n",
    "    dot_data = Path(out_file).read_text()\n",
    "\n",
    "graphviz.Source(dot_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overall Validation Performance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-24T17:38:08.706689Z",
     "start_time": "2018-10-24T17:38:08.660635Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 1104 entries, 0 to 215\n",
      "Data columns (total 4 columns):\n",
      "train             1104 non-null float64\n",
      "Validation AUC    1104 non-null float64\n",
      "time              1104 non-null float64\n",
      "Model             1104 non-null object\n",
      "dtypes: float64(3), object(1)\n",
      "memory usage: 43.1+ KB\n"
     ]
    }
   ],
   "source": [
    "models = ['catboost/factors', 'lightgbm/dummies', 'lightgbm/factors', 'xgboost/dummies']\n",
    "df = pd.DataFrame()\n",
    "with pd.HDFStore('results.h5') as store:\n",
    "    for k in models:\n",
    "        model = k.split('/')[0]\n",
    "        data = k.split('/')[1]\n",
    "        df = pd.concat([df, store[k][['train', 'valid', 'time']].assign(model=model, data=data)])\n",
    "df['Model'] = df.model.str.capitalize().str.cat(df.data.str.capitalize(), sep=' ')        \n",
    "df = df.drop(['model', 'data'], axis=1).rename(columns={'valid': 'Validation AUC'})\n",
    "df.info()    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LightGBM vs XGBoost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-24T17:55:45.268304Z",
     "start_time": "2018-10-24T17:55:44.626318Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABDAAAAFgCAYAAABNIolGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl4U1X+P/D3vUkXukELWNmFLlDWCogLi2yyVUBEkB1ERJ2R36iAAs4AIqLIiA6obF/ZXcANEXVgkE1AkCJlqaWlRaGiQEsLpXuTe35/hIRuaRPISdL0/XoeHprbm/M5N01Pbz5nU4QQAkREREREREREbkx1dQWIiIiIiIiIiCrDBAYRERERERERuT0mMIiIiIiIiIjI7TGBQURERERERERujwkMIiIiIiIiInJ7TGAQERERERERkdtjAoOcrqCgAD179rT6/cOHD+OFF16o8PmfffaZjKpVqnPnzmWOrVy5EidOnLD6nLFjxyIlJaXM8Y0bN1YYy9rz7NG6dWuMHTsWY8eOxbBhw/Cf//wHmqbdVpmVqez1ICLnYFtr4oy2tmfPnhg9erSlvX3uuefser4rX2sicpyq2u5evnwZPXv2tLSFRqMR48aNw759+6w+p2fPnigoKLD6/aVLl6Jv374YO3YsRo0ahYkTJ+LXX391eN2LS0tLw9y5c6XGINdjAoOqnLS0NLe60Zs8eTLatm1r9/OWLVsmoTYl1axZExs2bMCGDRuwefNmXLlyBR999JHUmLf6ehCRe2Fba5/Vq1db2tv33nvPrue622tNRK7hqrbgjjvuwOzZszFt2jQUFhbi3//+Nzp06IBu3brdVrkTJkzAhg0b8PHHH+OVV17Biy++WGHS43bVrVuXCYxqQO/qClDV8uWXX2L37t3Iz89HWloaxo0bhx9++AFnzpzBSy+9hN69e2Pr1q1Yt24dvL29cdddd2HevHkoLCzEtGnTkJWVhcaNG1vKS0xMxPz58wEAtWrVwoIFCyqtw/Lly5GcnIz33nsP+/fvx2uvvYaIiAjs3bsXe/bsQUhICM6ePYsrV64gKysL//znP9GxY0d8//33WLt2LVRVRYcOHTBt2rQS5b7yyis4f/685XHNmjVtugmdMWMGBgwYgE6dOuGll17C5cuXUa9ePRw5cgT79+8HALz//vtIT09HXl4eFi9ejG3btuHatWuYO3cuZsyYYfV5S5YsQWZmJry9vfHWW2/hzJkzWLlyJby8vHDx4kWMGDEChw4dwunTpzFu3DiMGjXKaj0VRcETTzyBWbNmYezYsejcuTMOHDgAAHjhhRcwYsQIXLhwodKf70MPPYS7774b586dw3333Yfr16/jxIkTaNq0KRYtWmR5Pe6//37MmTMH586dg6ZpeP7553HvvffinXfewaFDh6BpGmJiYjBhwoRKX2Oi6oZtbVlVpa0tbuPGjdixYwcMBgMCAwOxdOlSaJqGmTNn4s8//0RRURH+9a9/4YsvvrC81uPGjcP06dORnZ0No9GIf/zjH7j//vvx8MMP46677oK3tzdGjx6NhQsXQq/XIygoCP/+978REBBgU52IqHzVvd3t3r07Dhw4gL///e8wGAz48MMPAQAZGRmWxEbTpk1x6NAh/O9//wMAzJ49GxcuXEDt2rWxcOFC+Pr6Wr22sLAwtGrVCkePHsXRo0dRp04djBw5EikpKZg7dy42bNiAgQMHomPHjkhKSkLTpk1Ru3ZtxMbGwtvbGytXrsTy5ctx7tw5ZGZm4tq1axg1ahR27NiB3377DQsXLkSdOnXw4osvYvPmzfj555/xzjvvQKfToVGjRpg3bx7++OMPzJw5E3q9HjqdDm+99RZCQ0Mr/bmQmxFEdvjiiy/EE088IYQQYtu2beKxxx4TmqaJn376STz77LMiIyND9O7dW1y/fl0IIcTrr78uNmzYIDZs2CAWL14shBAiLi5O9OjRQwghxLBhw8SZM2eEEEJs3rxZLF68WBw6dEg8//zzVuuQmpoqhg0bJoQQ4vPPPxcLFy4UQggxZcoUcerUKbFkyRIxY8YMIYQQSUlJYuDAgSIzM1P0799f5ObmCiGEmDZtmti/f7/d1//AAw+UOfbyyy+LvXv3irVr11rqkpycLFq0aCGEEGLMmDFiy5YtQgghlixZIlauXFmirIqet23bNiGEEBs3bhQLFiwQhw4dEgMGDBCFhYXi2LFjolu3bqKgoECcP39eDBo0qNL6ZmVlib59+5b53vPPPy8OHTpU6c9XCCGioqLEhQsXRGFhoYiOjhZnzpwRmqaJHj16iGvXrllej48++ki89dZbQgghMjIyxIABA4QQQnTr1k2cP39eFBQUiE8++cSOV5+o+mBbW7Xa2h49eohRo0aJMWPGiDFjxojdu3cLo9Eoli5dKoxGoxBCiIkTJ4rY2FixZs0asWjRIiGEEImJiWLNmjUlXus333xTrF27VgghxMWLF0WPHj2E0WgUPXr0EPHx8ZZzVq5cKYxGo/jf//4nLly4YPdrTEQlVfd2VwghfvvtN9G8eXOxdetWy7HXX39dbNy4UQghxP79+y3X16NHD3Hs2DEhhBALFy4U69atK1HWkiVLxMcff1zi2Ouvvy62bdtW4nvJyclizJgxljJjY2OFEEL07dtX7NmzRwghxOjRo8Wvv/4qlixZIl555RUhhBArVqwQ/+///T/LazV//nzL66dpmujTp49IT08XQgjxzjvviE2bNomNGzeKefPmicLCQnHw4EGRmJh4S68TuRZHYJDdoqKiAACBgYEICwuDoiioWbMmCgoKkJqaivDwcEtP0D333GPp4eratSsAoF27dtDrTW+9lJQUvPrqqwCAoqIiNG3a1K66DBgwAEOGDMGTTz6JixcvolWrVti1axfuu+8+AEBERATS09Nx/vx5ZGRkYPLkyQCAnJwcpKamlijrVnsFzVJSUixD7cLCwhASEmL5XuvWrQEAderUQXp6us3P69ixIwCgffv22Lt3r+WavLy8EBgYiMaNG8Pb29vy+lfmwoULuPPOO8scF0JYvq7o5wuYehHq168PAPDz80N4eLjl/OJ1SEpKwtGjRy1z1g0GAzIzM7F48WIsXrwY6enplvcEEZXFtrZ87trWrl69Gj4+PiWOeXl54cUXX4Sfnx8uXrwIg8GAs2fPWuoRGRmJyMhI/PHHHyXqOXDgQABAaGgoAgICkJGRAQCWn9szzzyD5cuXY/z48QgNDeW0PSIHqc7tblFREWbMmIHZs2fjnXfeQadOnRAaGoqUlBQMGTIEwM22EjC1b9HR0QBMbad5VG9F/vzzT/Tp0wdnz561ek6rVq0AAEFBQQgLC7N8bW57W7ZsCcD0MzLfg5ZumzMyMnD58mU8//zzAID8/Hx07twZzz77LFatWoVJkyYhMDCwwvVIyH0xgUF2UxTF6vcaNmyIlJQU5Obmws/PDz///DOaNm0KRVEQFxeH3r1749dff4XBYABguhlbuHAh6tevj6NHjyItLa3S+KqqWhairFGjBu699168/vrrGDx4sOWc+Ph4DB48GElJSQgNDUXDhg1Rr149rF69Gl5eXvjyyy8tf6TMXn/99Vt5OSwiIyNx7Ngx9O7dG+fPn0dmZmaF55uTBhU97+TJkwgNDUVsbCwiIiIAVPz6V0TTNKxevRoxMTEATAmFnJwceHl5ITk52XJeZeXbGr9Zs2a488478cwzzyA/Px/Lli2Dv78//vvf/2Lx4sUQQiAmJgYxMTFo0KDBLV0TkSdjW1s+d29rzU6fPo2dO3fis88+Q15eHh599FEIIRAWFoaTJ0+id+/eSE1NxbvvvoupU6daXuuwsDDExsaiZcuWuHTpErKyslCrVi0App8JAHzzzTcYMmQIXn75ZaxYsQKbN2+2e+FQIiqrOre7CxcuRIcOHTBq1Cjo9XpMmzYN69ats7SdUVFRiIuLs5xfVFSEhIQEREVFlWg7rUlKSkJycjKio6Pxyy+/WF6P+Pj4Euc54j40ODgYd955Jz744AMEBgbihx9+gJ+fH3744Qd06NABzz33HLZt24b/+7//wxtvvFFpeeRemMAghwoJCcGUKVMwbtw4qKqKxo0bY9q0adDpdJg5cyZGjhyJZs2awcvLCwAwd+5cvPzyyzAajQBMDezly5crjFG7dm0UFRVh0aJFmD59OoYPH46RI0eWWLQnISEB48ePR15eHl577TWEhIRgwoQJGDt2LIxGIxo0aID+/fvbfX1Xr17Fo48+ank8ceJEy9ePPfYYZsyYgdGjR6N+/fpleuJKCwsLw7Rp0zB//nyrz9u5cyfWrVsHf39/LFy4EKdPn7arvteuXcPYsWOhKAoMBgMeeOABPPbYYwCAcePG4fHHH0fDhg0tIyocacSIEfjnP/+JMWPGIDs7G6NGjbL0YA4ePBg1a9ZE586dpcQm8nRsa92rrS1PkyZNUKNGDTz66KPw9vZG3bp1cfnyZYwYMQKzZs3CmDFjYDQaMWvWrBKv9dNPP41Zs2Zh+/btyM/Px7x58yw9umZt2rTBjBkz4OfnBy8vL8ybN++260tEFfPkdnfHjh04ceKEZaH34cOHY//+/fjggw/w1FNP4aWXXsL333+PO+64w9IeeXl5YcOGDTh37hzq16+PqVOnlil37dq1+O6776CqKvR6PZYsWQK9Xo/+/fvj+eefx5EjRywj5xxJVVW88sormDx5MoQQ8Pf3x1tvvYWcnBxMnz4dS5cuhaqqmDlzpsNjk3yKKD52nKgKOnHiBDZu3Ii33noLgGnbJvPCQM70yy+/IDc3F126dMHvv/+OSZMmYefOndKeR0TkTGxriYicyx3a3b179yI4OBht27bFwYMHsXz5cqxfv95p8YlK4wgMcltz58617Edd3KpVqyyrHG/cuBFffPEFlixZ4uzqldGoUSO8+OKLeO+992AwGDB79mypzyMicgS2tUREzlWV2t2GDRti1qxZ0Ol00DQNr7zyikvrQ8QRGERERERERETk9lRXV4CIiIiIiIiIqDJMYBARERERERGR2/O4BMaZM2dcXQUiIo/DtpWIyLHYrhIR2c/jEhjmvZeJiMhx2LYSETkW21UiIvt5XAKDiIiIiIiIiDwPExhERERERERE5PaYwCAiIiIiIiIit8cEBhERERERERG5PSYwiIiIiIiIiMjtMYFBRERERERERG6PCQwiIiIiIiIicntMYBARERERERGR22MCg4iIiIiIiIjcHhMYREREREREROT2mMAgInJzxuxCV1eBiIiIiMjlmMAgInJjhReuI/3/4lB44bqrq0JERERE5FJMYBARuSmhCWRtPwsAyNpxFkITLq4REREREZHrMIFBROSmcuMuwZhbBADQcoqQe/ySi2tEREREROQ6TGAQEdmhKNvolDjGnEJkH/wDMGgAAGHQkH3wDxhzipwS35GM2bmurgIREREReQAmMIiIbJT7ZyFS1qcj9y/5i2rmJ2YAotSUEU0gP/GK9NiOVPjnZWRs+BJFf112dVWIiIiIqIpjAoM8Xnpepqur4DRX8vJcXQWnyc1zzkgIM6EJ/LUrCwDw1w9Z0tej8G1eG1AUAEDwYy1MB1XFdLyKEJqG67sOAgCydh2E0DQX14iIiIiIqjImMMijnbpyBmN3zMSpK8murop08WlX8MS3O/BretXqob8VaelF2PZdJtLSnTedIvNkLgy5pqSJIVdD5km50yJ0/l4IeKAhoFfh1SAQdZ9tj4AHGkLn7yU1riPlnUyEdiOppuXmI+9UootrRERERERVGRMY5LGMmhFv/7IOALD4l7Uwas7tsXcmo6bh3dhfAADvHvkFRg/u6dY0gZ9jTVuK/hx7HZoTduYw5BqRdjgbwmB6LAwCaYezLQkNWfyiQ1F38t1QFAWKtw5+7UKlxnMkLTcPOT8fBww3XiODATmHj0PLrT6jhMjx0vNyXF0FIiIiciEmMMhjbT27G1cLTEP+Mwuy8M1ve1xbIYm2JZ/F1fwCAMDV/AJ8m/KbU+NfyXPeSIgzKXnIzzclaPLzNSSnyP9AnHUmH7iRJ6lzj7/pC3HjuESKqkD11iE37pIpiaEqUuM5Uv6Z3wFRKpEmhOm4B8vLSXd1FTzWqfS/MP67jYhP/8vVVSEioltgrYPNkzveyPGYwCCPlJF/DetPb0W+0bTYYr6xEOsSvkZmfpbT6uCsnsLM/HxsjD+NAqOppzvfaMTGUwnIzJf74dosIT0Hz3yXiIR0+debl6/hVHwublwqjEbgZHyuJaEhS1CEL6AAEZPqonbHGwkM5cZxKpdvxF2AoqLOM6NR55nRpoOKYjruoa5cPIUfNo1FxsVTTo2bnev5SROjpuGd2N0AgMWxe6rFza4zE8NERM6gU1W8uv9QmX86lR9JyXZ8t5BH2nvhCIw3en9HN38YAGAUGvZcOOKU+M7sKdx3/gK0G7tVjGzZHABgFAL7Ui9Ij23UBN6L/QMA8H7sBRglT+c4n1pg2Zije7eaAEwbdZxPLZAaV++nQ917A6B6mUZAKHoFde8NgN5PJzUuYFo81C861PJ1VaH61YB/p3Y3D+j18L+3HVS/Gq6rlESaZsTxH98GAMT9uBiak6as/XH5JFZtGY0Ll52bNHG2b1JO4Wq+abTV1fxcbEvx7Ot1ZmKYiIioKmECg5zKWT2F3Rt0gk4xvb3HtDAlMHSKiu4N7pEe29k9hd0aN4CqKPhk0ACMbNkCnwwaAJ2ioFujBlLjAsD3KVdwLd+0MMTV/CL8N0XuAqJNGvlAUYBHBoXgjrpeGPZobSgK0LiRj9S4ABDcxs80jUNRoPdTEdzGT3pMwDSN5Pqec7i+51yVmkICADXaNLd8rfr5okbr5hWcXbX9/utWFORdBQAU5GXi94RvpMfUNCN2HPo3AGD7oX87LWli5qwdnjLyc7Eh/gjyjaa2Jt9owPr4I8jMl7uQrqs4OzFcmrN3eCIiz1b6PnhOl/vK/KvofKLS9K6uAFUff1w+ic92TsPw3m+jwR2tpcYK9g3CuBaD0KdJZwDA5wPewf/OH0Swb5DUuED5PYWDI9pKixfs64sxrVqghpfp19nf2wsbB/WHt07u6IDM/CJ8Gn8JBUbTzXWBUeCT+Evo3KgWavnKaVp8fVW0buUHby8VZ1LyEBFWA21a+cHXV34uVlEViBvDP+r1CqpyyQRXUIoNCQ3q+UCJx54kPzcDib+sh9FgmrZlNOQj8eg6NGj2IHxqBEuLG5f0NXJuJE1y8jIRl7QV7VsMkRavuFNXzmD6/rexqMs0tK4dLjXW3tRkGEXJD/FGIbA3NRmPSGxbXaW8xHBMRB2nxE5LL8LuvdfQ48GaqFun6ux4RETuS6eqePq/P9h8/op+vSTWhjyBZ95NkttxRU/hoGY94O9VA1vP7oa/Vw0MbNpdekxX9RQ+HN4MqqJgZdxJrIw7KT15AQAHUq9hxYAW+Gxoa3w2tDXWDoyCJoADqVelxo0IqwEhBCLCTFMRIsKdNyXBPALDr56302JWZcWnvOhD67qwJnL9eXYvhCjZpglhxIWze6TFzMnLwMET62AwmpImBmM+Dp5YixwnjIpw9g5P3RuFQ6eYEoZvdBsIANApCh5sJDdx4grWEsNXbyQ0ZHLFDk9ERET2YgKDnKK8nkLZdKquwscymHsKv3l0Mr55dDKAmz2Fshg1zbL40eToNpgc3cZyXKZ+YbXh733zNfX31uHjIa3QN6y2tJiaJqCqpiSCmaIovNF2U4qqIHv/EWTvP+LRI1YahHWHougQ88S3iHniWwCAoujQoFl3aTETz+2xJIInDjIlEzRNQ+K53dJimjl7h6dgXz+MbXUPfHV6tKlTD5sHPYFxre5BsK9zpnEBwJU852z/eyD1GszN2SdDWmHdIOckhgHX7PBEROSprN2b8p719nEKSTHG7FzoApx3QwQAhuwr0AfI+8DnDqz1FDZv0h3+EodXA4AmNAxq1sPytarIzdl1bxSOjfElFwqV3VNoXtG5tNJzCh1NrypYcOBcmeOzOjeRFlNVFfx4oOxOMl07y58a5GxCE5YP/YHdm5R7nKxzZtvqUyMYzduPA2D6uej0vmjeYbzU6SPNm/TAwRPr8PSjn8LHOwB/e+wLrNoyGs2b9JAWE7C+w9ODDe6ROkVvYFhrfJMSD0VR4O/ljYfD5E5DLC4+7Qpm7t2PN7t3Qcs6ct9TXRrVwqfxl/DpkFbQqQr0qg6rYlqgSPINr7Udnho38nXKFD0iIk9g7mgDYPm/tOLHi59PtmMC44bCPy/j2tc7UOuRPvCqd4dTYuZdiMefX85C/aFvoEb9lk6JaYmdk44a/s6ZU2vuKXx+5PcAgHc/6W/pKWzf4lGpsVVFxbKTmwAAz7Z5XGoswNRT+PHA8VBvjBDYPHACdp5LdGpPITmetaSBzGSCoiq4uiWxzPFaj3juQpiO4oq29a6Wgywjg3xqBOOuqIFS49XwCcJzw7+2PPbxDsBzw7+WPj3PvMPTt4M+AADEbP2bZYenIWHy5i3rVBUr+gwHAGhCOG3LPaOm4d3YXwAA7x75Bcv69pIau5avHiNahUKnKlh93LSL1cR29aTFMyu+w5OZeYenyAj50/SKso3wCpA/SrIqEppm1/pB9p5PRI6jqgq+2277VM4BfeV25HoqJjBgauyv7zoIAMjadRAhIwdJb/yFZsTlnf8BAFze+R80HvMBFCdMcQCAKxdP4afvpuOBAYsQcqf8XixzT6G5dxIAVFWV3lPoKj46PVYcPwAAeLpdZ6f2FJIciqogdVvZIdyNHq7lgtpUDeab6IAu95Q5Jjeua9pWVdVB3Ni6Obrri1Alx1RVHb7a888yx4d0ny81bvcGnfBQ4wcso9k+G7AYY7fPcMoOT3pVh29STmGgE9vUbclncTXftE3z1fwCfJvyGwZFhEmN2V/iNDxrmjTywal405at3bvVxJ5915y2w1Pun4U4vyUTjYcEO319IWN2IXQB7r2mkaKqyPj468pPvCFk1GCJtSEiKsvZHX1MYADIO5kI7cb8Vi03H3mnEuHXNkpqzGvHv4Ux1/SByJhzFddOfIta0YOkxgRMi2ke//FtAEDcj4vRfegq6Tfa/jWC8UDb8QBM3Tt6nS8eaDtB+vQRd+GsnkJnMGoCumINUWXTRUqfX5WUbnQrS1Y4opEuXUZloy3ceTqJoqq49m3J9RhqxshPWrqqbQUA5caHelmJYU0zlmivK0tWlD7fEYK8A0q0aQFefvjq4SUeue1dZn4+NsafRsGNeRX5RiM2nkpA10YNEOzrKy2uK9pM8w5Pze7yhV6vQKeDU3Z4EprAX7tM0wL/+iELzUbVdlqbVnjhOjI/S0DwsCh4Nwh0Skwi8gwVTf1wxKiKqja1xNmjhqUkMDRNw9y5c5GYmAhvb2/Mnz8fTZrc/KCzd+9evP/++wCAli1bYs6cOVi1ahV+/PFHAEBWVhbS09Nx4MABrFmzBp9//jlCQkIAAK+++iqaNWvmuLrm5iHn5+OA4cawW4MBOYePwzf8Lqh+coZNGnIykXFoI4TB1KsjDPnI+OkjBER2g95Pbo/u779uRcGNxTQL8jLxe8I3aNbqEakxNc1YYqrIlMflD3N2FfOCmk+361zucUfHMatsvQtHxdepCqZsT7L5/KV9I287ZulGvLL1LhzV6CuqgrMfpdt8frPRtz8lS1EVpK89bvP5dSa0u+2YnsSVbaszqKoOa755wubznxi4xuF10KkqZv+0tMzxefdPcXgsV9t3/gI0IfDp4AHQqyoe+2objEJgX+oFDJY8CsMVIsJqWNpOX18V4WHyp45knsyFIdd0P2DI1ZB5Mhch7fylxxWaQNb2swCArB1nUXt8W7dNBhOR+1FVBZu/sP0e0V7Dhzpnmv/tcGWnm5QExs6dO1FYWIhNmzYhLi4Ob775JpYtWwYAyM7OxqJFi7B+/XqEhIRg1apVyMzMxOTJkzF5smnXhqeffhrTpk0DAMTHx2PhwoVo3VpOj1b+md8BoaH2xOFQfbyhFRTiyrovkH/md/i1kzMKIztpH4TQcNfkj6DzCQAAGAtycD3hB6k9hfm5GUj8ZT2MBtNimkZDPhKPrkODZg9KXWyuvKHOsoc5F//AXnztC0cnEkrTqSrmHPiuzPFXOw9weJzqsqc25xOSrcxtKwA0Hr8S59dNhhBGZCftc9ooDHK84u128ekjstvzbo0bYGN8Amrob94q6RQF3Ro1kBbTlVRVgbixEEanjoHSe/8MuUakHc6GuLFDrDAIpB3ORlCEL/R+ckeG5sZdgjG3CACg5RQh9/gl+N99p9SYZB+hGcud/mftOLmeQdOgt9ImO+I+tKLyyflc2ekmJYFx9OhRdO3aFQAQHR2NU6dOWb537NgxREZGYuHChUhNTcWwYcMsoysAYMeOHQgKCrI8Pz4+HitXrkRaWhq6d++Op59+2qF19Y24C74twqD6mOZAKl5egKLAN+Iuh8YpLqD5gwhs2cuSvAAAnY8/arZ17Ifc0v48uxdCGDFgwjaoqg6aZsR/NwzBhbN70KzVEIfGsmeos4xhztWpl5BuT0UZ4dsdVeHOUzw8UUDzB5Fx6CM0mfwxdD7+uGvyxzi3eiICIrtJj118fQ9PW0TPqBlLbENdWTta+vzb5azEcGnBvr4Y06oFzGtb+uh0GNM6Sur0EVczL0Rbt46X9FhZZ/LNM0vR/Jk7oBUJJK9NQ9aZfKmjMIw5hcg++AdgMCU7hUFD9sE/4BtZGzp/+ddNtlFUHQ7vmF3m+L195rmgNmQLvapi0Oe2r9dir62PcX0XVxFGAUVX9n7WEUkJa2VXREoCIzs7GwEBxT6c63QwGAzQ6/XIzMzE4cOHsWXLFvj5+WH06NGIjo5G06ZNAQArVqzA4sWLLc+NiYnBqFGjEBAQgOeeew67d+9Gjx7W51EXFBQgISHB5rpGRkRAV6x3RVEV1H1qBIwGg13l2CsqKgp/bS3ZCNcbNFtqzIgWMWjW+maiQlV1GDB+KwyGQofHjYqKsnmo8xMD1zgkfnhEOLz0N28+KrvJLjIUIflMsgPiRsCr2HuoopvqIoMByWfO3HbMqCj7Rwc54jV2RVzZMaOionD6g0stJzJlAAAgAElEQVR2x7BFi7+F2l0Xe1VW/q2UWR5b2tbS7WlFa14YDQYkOeB3oTS1aW+oXqah7zoffzSZtA5Jyb8B+MvhscwiIsKhL9b2mJMXBkMRztxCGxMeEQYvffkLCzpiWkiRoRDJZ1JsPj8qKgpP7vyXzed/2Pu12/69t6ddBRzXtpY2ICLCsqvU2pi++C35jNS/0+ERkfDSm5I/xXcfKTIYkXzG9ul7t8rcXsi8xpsUQNSCAgVQANVbgSYELiEVlxLkbRvr83sh/DQNof/vHiiqAq3AgMsrfsG5fb+i4C7bF/R0RNtqS7vqqr/3rlbRdXvC9XkiR91vVMRdfvbV6VoB0/VeevdnKWWHPt/Jcq22vq5SEhgBAQHIycmxPNY0DfobNyK1atVCmzZtULduXQBAx44dkZCQgKZNmyI5ORlBQUGW9TKEEBg/fjwCA02LKz344IP49ddfK0xg+Pj42P2mKr3QHGC68Xb0m7P0sLd6g8pmls0xHTVEzpbRDXq9tyWujNEQtnDUa23vTbaj4j61/RObzlvVd6RTGr3y2BPXoAnorYwcsHddi9JxrZXtyAWRyrtWVy2IJPvn7az3k61tq62r5YeMGiynjS1Vps7LFy2aRzp8yLFtbasXoqKibqldfefjvrdTvQq9MGp7ua+9QTNCb6WeH/Z+za4YZX/vrZdtja3tKiCnbS09RcXf2wutW7aUPnVlwYFzZY7N6txE+u+6KLaPaosWLSyjMWTKQA5qtqhhidV8UihUvdypHMbGRVB6KpbRcYqXDopORZNuLZ0+AuNW7llt4ar7jNtVuq20NtrC1fesjmatTZHd1lRFVfW9fSt4rdZJSWC0b98eu3fvxoABAxAXF4fIyJsfeFq3bo2kpCRkZGQgKCgIx48fx/Dhpr3dDx48iG7dbg71zc7OxsMPP4zvvvsOfn5+OHz4MIYOHSqjyk6hqDqc3/CMTec2HrvcITFVVYddnz9p8/k9H/vQrvKNmgE6tfy30e32FFor25E32eW5lRttWVw1n1CvKhj2xalynnH7Phta/no2XBCJbkXxZK+1JEXx445KDtvTttrTriYnJyMpKQnJp3IqP/kWJScnIzw8vMxxvapD/69t+xtlr+8Hl/83raI2blXfkbcd93bmTOtUFa/uP1TmeGWLJtvLHXZ3Kp3gNScUZCZ+hSbKTBVR9Yr0qXfmJMX1PaZEUWD3Jgh4oCGnj7gB2fes7qT4VDtrSYrixx09NY+oKpOSwHjooYdw4MABjBgxAkIILFiwAGvWrEHjxo3Rq1cvTJ06FZMmTQIA9OvXz5Lg+O2339C5883dGwIDA/HCCy9g3Lhx8Pb2xv33348HH3xQRpXpFulUvbSewhdGbS/3uMybbMD6jbYrcD6hZ6povp+r5hNWVfYkhgH7k8OaZoBqJUl7uzfPFZVdXehVFTFfrJBW/rdDHbtulgyu2N2pNFVV8OOBrDLHK9v16XYoqoLUbVfLHK9sy+rbZU6QBHa/mSjyv/tOrllEUljrFLM3GWHtfHfqdCNyFil3TqqqYt68ksO+wsJubjkWExODmJiYMs+bM2dOmWOPPPIIHnlE7jafRETOpOgUaXMJAdN8QpkqWqQyZNTtJcWslS00AxQrH/YdMWLNWvmqqse21f1vu/zyPDzx+zLHwsPDER4ejsSr/5ES0xyjNPPIj+w4OevBJLcqf9SHu5G5RbUjp+bZWz7dpKgKrm5JLHO8si0AyXFkJoYrK9/ZqlOnG5GzuMdvN9226vTHwFVkDnUur2zzB4qsE3G3VXZFkqNblflQYY579USqnJjtfKvEBxkqn+V9ueNHKeV3fO7Jct8fiqpHylJ5I4bCppQd6WS+1mMJ2VJitrAylYNcR+YW1TKn5gHWp+cRuRuZiWGg/OSwK8hODANVJzlM5EjV+xOpJDJ7Cl3RSwi4zx8DV5I51LkqDHMmIs9lHvmxtGirtPLdicz1hW5n3Q0iIiKqGBMYEsjsKSyvl5A8k/kDxfLcIqkxrMVdkyenp7Ci4esnT5Sdg+0o7dt1dLsPUVWR+f2Rdi5DSvl13ehnZL5W7eJSaeWTa8hcX6iqrC1UeoHOyta7cMSCnqXXmahsvQtHrUtRupzKpotwPQx5ZI9sA9xndJvsxLA5BlF1wwQGVSkcjkeOZH4/XUgqu5CcI+iTO/G9RGSF+ffv+omy6xE4LEY7tufWqKqC77Zn2ny+vdtZl0dRFZz9yPbdppqNdszuUYqqIH3tcZvPd8RiykTkmdjp5nrVMoFRepG4mjE97DqfiBzL3EuRnSdvG9XwcPfYRtWyxkhysrQYnZJD+IePiKgKkbk4cmXlO5PskW3mGETkGLLvW2/lnrVaJjAUVUXGx7YPHXXEHw6ZOBzP8TGoejC/n07/LmdET3h4qJRyiTyB+ffv/Tw5CwabYxBVBYqqIm3ZRmnl1312jLSyiaqT6tTp5q6qZQKDHMOcOEk+lSOtfHe5+ZQ91JnDnKsP8x++S0kSt1Hle4nIrcje3QngDk9EROR4su9bb+WelQkMD8DheETkycwf/i6ekTffVLhRwpTIkSpafNMR61o4YnFPIiIiWzGBQbfMnDhJvPofaeW7C9lDnd3pWonoJk0zSNtGWtMMUMvZFtuoGfDCqO1SYprL11nZ6rs6MCfEsk7EySk/ulWZNl327k7mGOVRVQWbv5A31Hn4UA51JqruZLerQPltK1VP1fcORhLZPYXsJSQiVxOaJm0+dXkLzZk//KWkrJASEwDCrH74s/5nctfnT9pUds/HPrSr7IqSC2u+ecKmmADwxMA1dpdPnqW6rZYvjAKKrvzRII7YWaSi8omIyDk8+i6muqzoTETkTBW1e7YukGytDWab6joGzYjvBy+XVrZe1ZVzXMO3Q5+WEtNcvt5KQmx5bpGUmO7yYd6VhCagWJlW4oitUa2Vr+gUXHpX4vpCz3eSVjZRVSa7XTXHIAI8PIHhihWdZfcUWuslJKKqQxiF1Bth9hLSrSgvwWD25M5/2VzOh71fs7ns0smF4p7a/onNMVf1HWl3+dWBq1bLV1QFpz+Qs7sTALT4G3d4IiKqrjw6gUFE5I4qSi6krz1ucznWhkQzeUFEREREjiCz4+1WOt2YwCCiak1oQlpvXkXDqD2R0DSbp+dxCh6RZzKvu3Eh6aq0GPrkThxOTkTkJNYSDK7qdGMCg4iqtYoSDGc/sn3YdXnzuqtT8gIou37FtW93l3hcM6aH1XMrIzQDwqbYtr7GrRCaAQoXtySqssyJk6vJydJidEoOYeKEiMjFeLdGRERur6LkwvkNz9hcTuOx5S9SyeQFkWOY1904/bu8NTDCw7kGBhFRdcU7NiIbyVwtv7yV8omIiMg25sTJpSSJu5Bw9AURkcsxgUFkI0esls+V8q3TNIHhQ29/e72Kyler2ZQOInIfBk3gs6GtpZavZxtHREQejgkMInILFSUXvtueaXM5A/oG210+EZFsFSUXpmxPsrmcpX0j7S6fiIgch51ursUEBhGVIbOnkL2ERERERFRVsdPNtTw2gWFejTorMV5ajI7JyVyNmjySzJ5CJi/IkYRmtLowp7XzFVV323E1zYiej31o87mqA2ISkWfiPSsRke08NoFBRESer3Qy4q+t88qcU2/QbKvn36rSCYnDO2aXOefePvPKPZeIiMjTGDQNWx8bLLV8rhlHgAcnMMyrUaedy5AWoy4z2S5h0Iz4frDtPa63Ur6eHziIiIjICXjPSp6gouTC0//9weZyVvTrZXf5VL14bAKDPFdFyYUnd/7L5nI+7P2a3eUTERFVJa5YbE5oAi3+FiotptAElHKmIwqjQOjzneTFNQooOk6DJCJyJSYwiIiIyCqjZrSa8LV2vo6JYLfhisXmyksumJ39KN3mmM1Gl594sVZ+RcmF9LXHbY5bZ0I7u8snIiLnYAKDiIioCtA0I54YuMau8x2x/kbpZMTsn5aWOWfe/VOsnk9ERETkKExgEBERVQGlkxFf7flnmXOGdJ9v9Xy6SeZic1xojoiISB4mMIjcGFd0JiJyPJmLzbFNJSIikocJDCI3xhWdiYiIiIiITPjphYiIiIiIiIjcHhMYREREREREROT2mMAgIiIiIiIiIrfHNTCIiIjIrRg1Dav6jrTrfB3X9CEiIvJ4UhIYmqZh7ty5SExMhLe3N+bPn48mTZpYvr937168//77AICWLVtizpw5AIBu3brhrrvuAgBER0dj6tSp2LVrF95//33o9XoMHToUw4cPl1FlIiIichOlkxFzDnxX5pxXOw+wej4RERF5JikJjJ07d6KwsBCbNm1CXFwc3nzzTSxbtgwAkJ2djUWLFmH9+vUICQnBqlWrkJmZievXr6NVq1ZYvny5pZyioiK88cYb+Pzzz1GjRg2MHDkSPXr0QN26dWVUm26BUTPghVHbpZWtUzlIiIiIiIiIiCQlMI4ePYquXbsCMI2kOHXqlOV7x44dQ2RkJBYuXIjU1FQMGzYMISEhOHToEC5duoSxY8fC19cXM2fORGFhIRo3boyaNWsCADp06IDY2Fj0799fRrXpFlSUYFjzzRM2lfHEwDV2l01ERERERETVi5RPiNnZ2QgICLA81ul0MBgM0Ov1yMzMxOHDh7Flyxb4+flh9OjRiI6ORt26dTF58mT0798fsbGxmD59OmbOnInAwEBLOf7+/sjOzq4wdkFBARISEgAAUVFRMi6vBHOs4mTHdUVMa3Gtsbc+9pTtqJiuiludrtURcXmttxfTUW1D8bbVFtbiOuJnY29MT4zLa5UT01N+7901riddqyPaVne4Z3UFXqtjucu1WuOq33tX4LVWzFHtqpQERkBAAHJyciyPNU2DXm8KVatWLbRp08YyDaRjx45ISEhAjx49oNPpLMcuXbpUppycnJwSCY3y+Pj4OKWxMHNmLFfGlB3XE6/JnWJWt7i8VsdzVNtanX42rorLa/X8OvBn7Blxq8M9q6vwWqs2T7wma3it9pOy6lX79u2xb98+AEBcXBwiIyMt32vdujWSkpKQkZEBg8GA48ePIzw8HO+99x7WrVsHADh9+jTq16+PsLAwnDt3DlevXkVhYSFiY2Nx9913y6gyEREREREREbkxKSMwHnroIRw4cAAjRoyAEAILFizAmjVr0LhxY/Tq1QtTp07FpEmTAAD9+vVDZGQkJk+ejOnTp2Pv3r3Q6XR444034OXlhRkzZuDJJ5+EEAJDhw5FaGiojCoTERERERERkRuTksBQVRXz5s0rcSwsLMzydUxMDGJiYkp8v2bNmli5cmWZsnr27ImePXvKqCYRERERERERVRHcOJ2IiIiIiIiI3B4TGERERERERETk9pjAICIiIiIiIiK3xwQGEREREREREbk9KYt4kvNpmgEPT/xeavmqyrcLERERERERuQY/kXqIipILuz5/0uZyej72od3lV3dGTcOqviNtPlencuATERERERGRvfiplOg2lU5IzDnwXZlzXu08oNxziYiIiIiIyDZMYBARERERETmYQTPi+8HLpZavV3XSyidyR0xgEBERkU2MmoZ5908p9zhHmBERlVRRcuHJnf+yuZwPe79md/lEnop3G0RERGQTc5Ji2clNWHZyU5njRERERDJxBAZ5DKNmtJqhtna+jplrIiK6wahpWNGvl13nM3lDRETkPExgkMconYyY/dPSMucUH/rM5AURERVXOhnx6v5DZc6Z0+U+q+cTERGRXExgEJHNjJrA0r6Rdp2vUxWJNSIiIiIiouqCCQwislnpZMSCA+fKnDOrcxOr5xMREREREd0qjn0kIiIiIiIiIrfHERhERERELsKpeURERLZjAoOIiIjIRTg1j4iIyHacQkJEREREREREbs+jR2AITUPdZ8dILV8pZws1oRkQNuVrSTENUFSP/rERERERERERleHRn4TLSy6YZXxse4IhZNRgu8qvKMFwfsMzNsVsPHa53WUTEREREREReSpOISEiIiIiIiIit8cEBhERERERERG5PSYwiIiIiIiIiMjtMYFBRERERERERG6PCQwiIiIiIiIicnvc0oKIyE0ITaDOhHZ2na+oisQaERERERG5DyYwiIjcROlkxNUtiWXOqfVIc6vnE3kio6bh1c4Dyj2uq2C7dCIiIvI8/MtPREREbqt4kmLF8QNYcfxAmeNERERUPXAEBhEReQShGVFv0OxyjyuqzgU1IiIiIiJHYgKDiIg8QvEkRfq+VQCAOt2e8sjkhaYZMaT7/HKPqx54vUREREQAp5AQERFVOcWTFLtjP8Du2A/KHCciIiLyNExgEBEREREREZHbYwKDiIiIiIiIiNyelDUwNE3D3LlzkZiYCG9vb8yfPx9NmjSxfH/v3r14//33AQAtW7bEnDlzkJ2djenTpyM7OxtFRUWYMWMG7r77buzYsQNvvfUW6tWrBwCYMmUKOnXqJKPaRERERABM27TO6XJfuce5AwoREZFrSElg7Ny5E4WFhdi0aRPi4uLw5ptvYtmyZQCA7OxsLFq0COvXr0dISAhWrVqFzMxMbNy4Effddx8mTJiAs2fPYurUqfjqq68QHx+P6dOno2/fvjKqSkRERFSGOUmxMu4kAGBydJsSx4mIiMj5pCQwjh49iq5duwIAoqOjcerUKcv3jh07hsjISCxcuBCpqakYNmwYQkJCMGHCBHh7ewMAjEYjfHx8AADx8fFISEjAunXr0LZtW0ybNg16PTdPISIiqk40oeHpdp0tX6sKEwm3Q9MEBvQNtut8VVUk1oiIiKhyUjIB2dnZCAgIsDzW6XQwGAzQ6/XIzMzE4cOHsWXLFvj5+WH06NGIjo5G06ZNAQBpaWmYPn06Zs2aBQDo3LkzevfujYYNG2LOnDn49NNPMWbMGKuxCwoKkJCQUGH9oqKi7L6mysq0hb1xXRHTVXEdEdPW+I6O5Yq4rvq52loHT7hWV7/Gjnp9b+U6ymNL22pLXJm/f9Ziy47pqvamvNie+PoWj/tNyikMDGuNxNOJTotZnKe+l6KiovDjgawyx7t2DmJ77sAySyverjqqra6Is35XK1OdrtUaV99jOBOvtWK8VvvLlJLACAgIQE5OjuWxpmmWURO1atVCmzZtULduXQBAx44dkZCQgKZNmyIxMREvvvgiXnrpJcs6F0OHDkVQUBAAoFevXti+fXuFsX18fKQ0jM5obN0hpqviOjNmdXpdXRW/Ol2rK+K64voc1bZ6evvi6tie+voaNa3E48jmzV0ylaM6vZdcHdfV8Z0RV9Y9qzWu/lk6kydeqydekzW8Vs/kqGuV8te/ffv22LdvHwAgLi4OkZGRlu+1bt0aSUlJyMjIgMFgwPHjxxEeHo7k5GT84x//wNtvv40HH3wQACCEwKBBg3Dx4kUAwE8//YRWrVrJqDIRERG5qW9STiGnqAADw1pDEwLbUk5V/iQiIiLyOFJGYDz00EM4cOAARowYASEEFixYgDVr1qBx48bo1asXpk6dikmTJgEA+vXrh8jISDz77LMoLCzE66+/DsA0imPZsmWYP38+nnvuOfj6+iIsLAzDhw+XUWUiohKEJtBsdB27zlc4P5yqAU1oeLbN45avZa9FkZGfiw3xR7DqxE/Y9uhk5BmKsD7+CLo1Ckewr5/U2ERERORepCQwVFXFvHnzShwLCwuzfB0TE4OYmJgS3zfvUlJaly5d0KVLF8dXkoioAqWTEanbrpY5p9HDtayeT+SpiicsnLGQ5t7UZBiFAAB8nHAUHycchZeqw97UZDwS0VZ6fCIiInIfXMKbiIiIbsmpK8nSY3RvFA6dYkoQfpxwFACgUxQ82ChcemwiIiJyL9yPlIiIiGxm1IzQqToAwOJf1mJVr1ctj2UI9vXD2Fb3YEP8EeQbDfDR6TGu1T0eOX3EqAnM6tyk3OM6iaO8NE2ga+egco9z61QiInInTGAQERGRzbae3Y3BYT0BAJkFWfjmtz14JKyX1JgDw1rjm5R4XMzJQoivHx4Oay01nqsUT1KsPv4XAGBiu3pSkxcALEmKY3GmHeTujvYvcdxRuLYQERHdLk4hISIiIptk5F/D+tNbAQAKFOQbC7Eu4Wtk5mdJjatTVbzQsTsA4IWO3V2yhSrdvvLWFir9r6LziYiIeAdARERENtl74QiMQkPM1r/ho8RvAQBGoWHPhSPSY7euUw/rBoxBqzr1pMciIiIi98QpJCSFphnxxMA1Np+rSpw/TUREjtG9QSesT9iKIgAfJW4DAOgUFd0b3OOU+HVq+DslDhEREbknJjBIitIJia/2/LPE4yHd51s9l4iI3FOwbxDGtRiE9ae3It9YCF+dN8ZHDUawb9kFIIkqIjRRYivq4sc5dYSIiKxhAoOIyA0JTaDWI83LPc6be3KlQc164Jvf9uCv3HQE+9TEwKbdXV0lqoKstWMy2jehCdSZ0M6u89nOEhFZ58p2lQkMIiI3VLyRv77nHAAgsHsT3lSTy+lUHV5sPx7T97+NF9uPl7qFqqtpQmBydBsAQG5REXx0Oi4g6mC5fxbCr7631Bil282rWxLLnFM8Ycx2loioYsXbSWvJieLHHdmu8q8wERER2aV17Qhs6PMGWtcOd3VVpPrmTIrl6wnbtuPblN9cWBvPIzSBv3ZlQQhh+qcJV1eJiIjs5MwRdQATGERERHQL6tQIdnUVpMrMz8fG+NPQhIAmBPKNRmw8lYDM/HzpsTUhMLFdPUxsVw+a8NwP9Zknc2HINUJRFCiKgsyTua6uEhERuTlOISEiIqqCNM0IKAp6dPyb5TEXRXacfecvQBMCj3yx1XLMKAT2pV7A4IgwqbFVRcH3KVfQP6w2VMUzpzMYco1IO5wNYQCEENAKBdIOZyMowhd6P76PiYiofByBQURUCfNq+aX/cbgzuVJc0tdQFRVxSV/feLy1kmeQPbo1blAmeaBTFHRr1MBFNfIsWWfygRtN6JXYHJz5MA0QN44TERFZwQQGEVElzHP4Lv14HZd+vF7mOJGz5eRl4OCJdSgozEZ05GAUFGbj4Im1yMnLdHXVPEawry/GtGoBH51pNICPTocxraMQ7Ovr4pp5hqAIX+BGE5p+JMf0hXLjOBERkRVMYBARuTGhCQR2b4LA7k044oMsEs/tgaYZ8cHnQ3Et+yI++HwoNE1D4rndrq6aR3k4vBmCfX0AAMG+PogJa+riGskjhMDd0f64O9ofwgnrbuj9dKh7bwCUG5OZFb2CuvcGcPoIERFViGtgEBG5MUVVkBt3CX7RoRzxQRbNm/TAwRPrYNSKsHrreACAqqpo3qSHi2vmWXSqin90bI+Ze/fj+XvaO20LVU0I9A+rDU0Ip62BoRSLozgpZnAbP2SezENRlhF6PxXBbfycEpeInMeoaVjRr5dd53O7aqoI3x1ERERVjH+NYDzQdjz0OtNwe73OFw+0nQB/D98ZxBVa1a2NNTF90LJObafFNCctXLWAZ1p6kVPiKKqCej2DAAD1egUxSUvkgexNRjB5QZXhCAwiIqIqKDpyMOKStuJa9l/wrxGM6MhBrq6Sx6pdo4ZT4yWk5yCqjr9TY2qagHojgfBz7HX07xNseSyTX31vhI2rA68A50wdEZpArUeal3ucCRQiIvfHBAZRFcTheESkqjr0uW8qPts5DX3vm8YtVD2EURN4L/YPvNc3EvkGDd46FTonfLA+k5KHyHBToiY/X0NySh4iI5wzpcNZyQvg5uLL1/ecAwAEdm9S4jgREbk3JjCIqqDSyYhX9x8qc86cLvdZPZ+IPEPDO9rgqUc+QoBfHVdXhRzk+5QruJZvgKIoeOrb0xjZKhQxEXJ/vnn5Gk7F56LZXb7Q6xUYjcDJ+Fw0buQLX1/P+/thXhzZ/DWTF0REVYfVBEaXLl0AAEVFRcjLy0O9evVw8eJF1K5dG7t27XJaBYmIiMg6Ji88R2Z+ET6Nv4QCo8DmXy+jwCjwSfwldG5UC7V85fU5nU8tgBDAV1sz0L1bTQCAEKbjkRHOnT7jDOaEhXmBZCIiqjqsptX379+P/fv3o2vXrti+fTu2b9+OHTt2oG3bts6sHxEREVG1cCD1Gsy7JX+WcBkAoAngQOpVqXGbNPKBeb3QPfuuAQAUBWjcyEdqXCIiIntVms7/448/UK9ePQBAaGgo/vrrL+mVIiL3Z9QEZnVuUu5xR87X1jSBAX1t31mh+EJ0jiQ0gdCugZavnTXkWAgBv+hQCCGctrUhEblGl0a18Gn8pRLHVAXo3KiW1Li+vipat/LDqfhcGI2ATge0aeXnkdNHiIioaqs0gREWFobp06ejbdu2iIuLQ4cOHZxRLyJyc+YkxerjpqTmxHb1Shx3lNLJiB8PZJU5p2vnIKvnO0rxhIUz50ubkxZMXhB5vlq+eoxoFWqZRuKjUzCyVajU6SNmEWE1kJySj5wcDb6+KsLDPG/qCBERVX2V/kV87bXXsG/fPpw5cwYxMTHo2bOnM+pFREREVO30D6uN/6Zk4FJOIWr5eqFfWG2nxFVVBZ06BmL33mvo1DHQKVuouhJHtxGRo7nLqGFPZ3Vs4O7duwEAn332GS5duoSgoCCkpaVh06ZNTqscERERUXWiUxX8vWMDAMDfOzZwyhaqZnXreOHhAcGoW8fLaTFdhaPbiMjR7E1GMHlxa6yOwLh61bRgVFpamtMqQ0REJRVeuA6v+gGABig6/qEjqg6i6vhj+YDmqF3D+YkEvxo6p8ckIiKyldUERnR0NH777TfExMQ4sz5ERHSD0ASytp9FnYntkL7+OGqPb+vU9TeIyHVckbyoToRRACpQ9Gc2vBsEuro6RERkI6sJjNmzZ5d7XFEUrF+/XlqFiIjIJDfuEoy5RQAALacIuccvwf/uO11cKyKiqk1oAlfWnUCdie2QteMsk8NERFWI1QTGhg0byj1eWFgorTJERO5MCAHAOXOmjTmFyD74B2DQkH3oAoRBQ48xUFcAACAASURBVPbBP+AbWRs6f/bMVkRoGup0e8rytaJyK0giuonJYSKiqqvSXUg+/fRTrFmzBgaDAUIIeHl5Yfv27c6oGxGRW3HmYm/5iRnAjYRJzqELpoOaQH7iFfi35412RRRVxbXj36JmuxgmL4ioBCaHiYiqtkoTGJs3b8aGDRuwbNky9OvXD+vWrXNGvYiIytA0ga6dg8o97mkrOfs2r226yYa4eVBV4NvcOVsqEhF5IiaHiYiqtkq7poKDg3HHHXcgJycH9957L65du+aMehERlVE8SXEsLgfH4nLKHJdJCAEhBHL/kj+VTufvhYAHGgJ6UzOt6FUEPNCQPYRERLfBt3ltoPRoOiaHiYiqjEoTGIGBgdi5cycURcGnn36KjIyMSgvVNA2zZ8/G448/jrFjx+LcuXMlvr93714MHz4cw4cPx9y5cyGEQH5+PqZMmYJRo0bhqaeessTZtWsXhg4discffxybN2++xcskIro9QhNQFAWKouCvH7IgNFH5k26TX3SoJWGh+nvBr12o9JhERJ6MyWEioqqt0gRGRkYGkpKSMHXqVPz++++YO3dupYXu3LkThYWF2LRpE6ZOnYo333zT8r3s7GwsWrQIy5cvx+bNm9GgQQNkZmbik08+QWRkJD7++GM88sgj+OCDD1BUVIQ33ngDq1evxoYNG7Bp0yakpaXd1gUTEd2KzJO5MBZqEELAkKsh82Su9JiKqiCoTzMAQFCfZlwl30ZCaKjZLgZCaK6uChG5ISaHiYiqrkrXwJg+fTq++uorPPXUU+jduzcaNWpUaaFHjx5F165dAQDR0dE4deqU5XvHjh1DZGQkFi5ciNTUVAwbNgwhISE4evQoJk2aBADo1q0bPvjgA6SkpKBx48aoWbMmAKBDhw6IjY1F//79b+liiYhuhSHXiLTD2bh8IBt17vGHMAikHc5GUIQv9H46qbG9GwSizqRo6AK8pcbxJIqilvhfNqEZcW+feeUeV1S57w8isp85OZz5WQKTw0REVUylCYw2bdqgTZs2uHbtGubOnYs+ffqUSEiUJzs7GwEBAZbHOp0OBoMBer0emZmZOHz4MLZs2QI/Pz+MHj0a0dHRyM7ORmBgIADA398f169fL3HMfDw7O7vC2AUFBUhISKjwnKioqMouu4zKyrSFvXFdEdNRcW2pg6Pj2BLTE+O66lrLiy07nsvipvoARj8oUJB+xLTuhqYJnNl/DmhUIDf2bbqVNqA8trSttsR11nvE2aKionDq0DIAQOv7nvXY6yTyJMqD/sjI+gPIsv+5jmhbi7erjmqrK+Iu7VJ1ulZr3OH+nMjd2Pp7UWkCIzY2Fl9++SVOnjyJfv364eWXX6600ICAAOTk5Fgea5oGvd4UqlatWmjTpg3q1q0LAOjYsSMSEhJKPCcnJwdBQUFlysnJySmR0CiPj4+PlIbRGY2tO8R0ZlxPvz53iOvsmJ762hqaGJHyezqE4eYxVVUQ1qWJ9BEY7sJRbaur3iPOVl2uk4hunax7VmuqU7vkidfqiddEdCsqHV+7bt069OvXD1u3bsXf//533Hln5VtMtW/fHvv27QMAxMXFITIy0vK91q1bIykpCRkZGTAYDDh+/DjCw8PRvn177N27FwCwb98+dOjQAWFhYTh37hyuXr2KwsJCxMbG4u67777VayUiuiV6Px3q3hsA5UbKV9ErqHtvQLVJXhARERERuYNKR2AsXbrU7kIfeughHDhwACNGjIAQAgsWLMCaNWvQuHFj9OrVC1OnTrWsd9GvXz9ERkaiUaNGePnllzFy5Eh4eXnh7bffhpeXF2bMmIEnn3wSQggMHToUoaFcaImInC+4jR8yT+ahKMsIvZ+K4DZ+rq4SEREREVG1UmkC41aoqop580ouaBYWFmb5OiYmBjExMSW+X6NGDSxZsqRMWT179kTPnj1lVJOIyGaKqqBezyCc35KJer2CuOgbEREREZGTSUlgEFH1oAmBie3qWb5WFc/+UO9X3xth4+rAK4BTR4iIiIiInM05e8wRkUcqnrDw9OSFGZMXRERERESuwREYHk7TjOj52Id2na+q/IBGRERERERE7oUJDA/3/9u79+io6rvf45+5kGRIIhCgPi4FSjBImoCIoFEB4eH2VMBYGyEECKhFpYjnIJ4GqCBijFCrraLwQLFYgyCKPJWqrYiHioBQJCsthITKVQ8qBsFLLuQy+3f+iIwgF7nM3nuSvF9ruZz9m539/U003/zmk733fD+M2Lx6xkn7XDvwu/uVEF4AAAAAACIRAQYAAADgImNZaj1+lK3H93i5chxA/UeAAQAAALjoTOHC4aWvnfVxErLSz/n4AFCfEGAAqHeMMbqqa2zosaeR3EAUAAAAaMyIYwHUOx6PRx/urgw9BgAAANDwEWAAqFcsy5xxGwAAAEDDxCUkAOqVD3dXqv2PY5TUISBjjHbtrlTHpKZuTws4AR9JDQCnZlm1GnLHX209vtfLWxygoeKnG8AFsUzdGRBeBy7lqDxqaXtRhf75rwrddmtL1dYabSuqUNs2MYqJ4YQyuCdYW6XUtPGSpJrqMn384Rolptzi8qwAIPKcKVz4vyvuPOvj/GfGc+d8fAD1Hyt+ABfE6/E4El5I0kcfV+nbvEQ7iiv1P6sOy5i6ccAtRysOa/XSzND2W0tu086tf1JV5REXZwUAANDwEGAAqDfatYnWsaykqLhCkuTxSG3bRLs4KzR2n+x5V8YEZYwlYyxJkjFBHdjzd3cnBgAA0MAQYACoN2JivEpNaSrft7cW8PmkzilNuXwErrq0Qx95PD69sXiw3lg8WJLk8fh0aWIfV+cFAADQ0LDqB1CvJHUIhAKLmBivLu8QcHlGaOyiAy10Rbds+fwxkiSfP0ZXXD1G0YEWLs8MAACgYeEuNwAuiPn2phQeh+6D4fV6dE33eK199ytd0z1eXq8zdYEz+fFPbta+4r+o4ptPFR1ooR8nD3V7SgCACBW0gnqu/yPntL+PT7YCJHEGBoAL5PF4HAsvjmndqomG3NRCrVs1cbQucDper09X9rpfktS11/18hCoA4LTONYwgvAC+wxkYQD0XtCw91DPtlOM+r3MZZfGhciW3inWsXtMAv8wRWVr+R6r6Dc9XILaV21MBAABokDgDA6jnjg8pFhZu08LCbSeN2yVoGVXWBmWM0bMfHFDQMrbXBCIZ4QUAAIB9CDAAnLe/7v5C414v0SvFpfryaI3+tvsLt6cEAAAAoIEiwABwXo4crdFLRQdVFTR6pfhzVQWNlhUd1JdHa92eGgAAAIAGiAADaACCluV4zQ0ff6XvXzFiGWnDx186PhcAAAAADR838YTtLCuon/XJPWmMu/SHz+u79ii94+WO1uzZprleKjp4wpjXI93Qprmj8wAAAADQOHAGBmx3LKhY+8E8rf1g3gljuHBHjh7VkqKSU47bqXmMX5kpFyvaV/cRqtE+j0akXKzmMeSiAAAAAMKPdxpAGAUtSw/fcNMpx+36VJB1Hx2QZYyClqW7unaWJJVX12jdxweUntTBlprH/LRDS/1t92EdLK9W85gm+q8OLW2tBwAAAKDxIsAAwuhYSLHgnxskSXdfecMJ43bo3fZSLSkq1s9W/iU0FuPzacFP+9tW8xif16MJ3S/VjHf3akL3S+XzemyvCQAAAKBx4hISoJ5rEROjUSmdFO2ruywn2ufTqNRktYiJcaR+cqtY/fdNVyi5Vawj9QAAAAA0TgQYQAMw5PJEtYiJliS1iInW4A7tHa3fMtDE0XoAAAAAGh8uIXGQsYJqO/q/z3pfDze6xFnyeb36X927aeq76/W/e3Sz9ZIVAAAAAHADAYaDvh9IfLpq1gnbl9w847T7Aj8kpXVLLR48UC0DAbenAgAAAABhx59pgQaE8AIAAABAQ0WAAQAAAAAAIh4BBgAAAAAAiHi23APDsizNnDlTO3fuVFRUlHJzc9WuXbvQ87m5uSooKFBsbN3HLs6bN09PP/20SkpKJEmlpaW66KKL9PLLL59y3/j4eDumDQAAAAAAIpQtAcaaNWtUXV2t5cuXq7CwULNnz9b8+fNDzxcVFWnRokVKSEgIjf3617+WJNXU1CgrK0uPPPLIafcFAAAAAACNiy0BxtatW9WrVy9JUteuXbV9+/bQc5Zlaf/+/ZoxY4YOHTqkjIwMZWRkhJ5fsmSJbrjhBl1xxRU/uO+pVFVVqbi4+Iz7JCcnn/Nr+qFjnqvTzSHcdc6mpt11T1Xb6Xpu1ra7HnAm59PvTuVseuvZ1OXnAUBDEI7eWl/WrE5pTK8VwMnOtgfYEmCUlZUpLi4utO3z+VRbWyu/36+KigqNGjVKt99+u4LBoLKzs5WamqpOnTqpurpaL730klasWCFJZ9z3dKKjo8O2YD+eHcd0s47bdd16nW7UdvO1AuESrt7KzwMA1Knva9ZI0JheK4A6tgQYcXFxKi8vD21bliW/v65UIBBQdna2At9+3GNaWppKSkrUqVMnvf/+++rRo0foHhdn2hc4k6BladZ1E0857vNy71oAAAAAqG9seSfXrVs3rVu3TpJUWFiojh07hp7bt2+fsrKyFAwGVVNTo4KCAqWkpEiSNm7cqN69e5/VvsCZHB9SzN+2XPO3LT9pHAAAAABQf9hyBsaAAQO0YcMGZWZmyhijvLw8LV68WG3btlW/fv00dOhQDRs2TE2aNFF6erqSkpIkSXv37tUtt9wSOk6HDh1Ouy8AAAAAAGg8bAkwvF6vZs2adcJYhw4dQo/HjRuncePGnfR1CxcuPGnsdPsCAAAAAIDGg/PpAQAAAABAxLPlDIxIZyxLCVnp57S/h3snAAAAAADgmkYZYHw/jPjqjbUn7dNscN/T7g8AAAAAAJzFO3MAAAAAABDxCDAAAAAAAEDEI8AAAAAAAAARjwADAAAAAABEvEZ5E0/ATkFj6e4rb5AklddUK8bnl48bwQIAAADABeFdFRBmf9m1PfQ4+418vb57+xn2BgAAAACcDQIMIIwOH61QftGW0PbRYK1eKNqiI0crXJwVAAAAANR/BBhAGL378S4FjZH17T+SFDRG7368y+WZAQAAAED9xj0w0GAFraB8Xp+jNfu0uVxLirZo6MqFoTGfx6Mb21zu6DwAAAAAoKHhDAw0WKv2rJVlLI3vPFzjOw9XVbDa9potYppqdEoPxfjqssFon1/ZKT3UIqap7bUBAAAAoCEjwECDdPjoV3qhZJW8Hq9W7VkrScr86wM6cvRr22sP7ZCq5t8GFgkxTTWkQ6rtNQEAAACgoeMSkkbEWEFdO3DWKcc9Dl9qYbd3D2xR0Fgqr6nQzYl9VV5ToaCx9PcDW/SzDv1sre3zejWpex/lvLtKk7r34SNUAQAAfoBlBfWfGc+d0/7eBrZ+BfDDCDAakeNDiu2b5is1bfxJ4w1Fn0uv0QvFq5Tx5v2hsRhflPpc2sOR+qmtLtGfbhqlVoFYR+oBAADUZ+caRhBeAI0TfxpGg9Qi5iJld7pZMb4oSXXhxZjkdLWIucixORBeAAAAAED4EGCgwbo5sa9aRNcFFi2im2lo+z7uTggAAAAAcN64hMQlxgrqkptnnDTWEC/ncIvP69P93cbo/6x/Qvd3G+P4R6oCAAAAAMKn0QcYxrLUbHDfU457bLz54rGg4tC6P0iSWvUe16DDC8tY6tv9l6HHXo8zJ/+ktkxS/sDH1CrQwpF6AAAA4WIsSwlZ6ee0v53rVwBwW6PvcMeafNn6LSpbv+WkcYTH8YGFU+HFMYQXAACgPjrX9SjrVwANHV0OAAAAAABEPAIMAAAAAAAQ8QgwAAAAAABAxCPAAAAAAAAAEY8AAwAAAAAARDwCDAAAAAAAEPEIMAAAAAAAQMQjwAAAAAAAABGPAAMAAAAAAEQ8Agw4xhhLxlhuTwMAAAAAUA/53Z4AGg+Ph7wMAAAAAHB+eEcJAAAAAAAini1nYFiWpZkzZ2rnzp2KiopSbm6u2rVrF3o+NzdXBQUFio2NlSTNmzdPwWBQgwYNUseOHSVJ/fv315gxY/Tyyy/rpZdekt/v1/jx49W3b187pgwAAAAAACKYLQHGmjVrVF1dreXLl6uwsFCzZ8/W/PnzQ88XFRVp0aJFSkhICI1t3LhRQ4YM0fTp00NjpaWlys/P16uvvqqqqiplZWXphhtuUFRUlB3Ths2MMZIkj8fj8kwAAAAAAPWNLQHG1q1b1atXL0lS165dtX379tBzlmVp//79mjFjhg4dOqSMjAxlZGRo+/btKioq0qhRo5SQkKAHH3xQ27Zt01VXXaWoqChFRUWpbdu2KikpUZcuXU5bu6qqSsXFxec03+Tk5BO2z/Xrz1ck1HWq5vF1N255Qy3iEh2rCzRm3+8z5+tce+vp6jrZcwDALuHoreezZgWAhups+6otAUZZWZni4uJC2z6fT7W1tfL7/aqoqNCoUaN0++23KxgMKjs7W6mpqUpMTFRqaqquv/56rVq1Srm5uerXr5/i4+NDx4mNjVVZWdkZa0dHR1/wL5VwLfjPxKqtdqWuJBnLUmraeBnLcqymZQVDZ2CUfPqKxg55Tl6vz5HaAC5cOHqr5FyfA4BIF66+CgCNiS038YyLi1N5eXlo27Is+f11WUkgEFB2drYCgYDi4uKUlpamkpISpaWl6dprr5UkDRgwQDt27DjpOOXl5ScEGvXZwb/9VsGqMrXqPU6teo+TVV3pWG2P16u9O1bJ43XuHq6F/35NNbWVqqmtVHnlERX+e5VjtQEAAAAA9Z8t72C7deumdevWSZIKCwtDN+aUpH379ikrK0vBYFA1NTUqKChQSkqKHnzwQb311luSpPfff18pKSnq0qWLtm7dqqqqKn3zzTfavXv3Cceqr6pK96pi72btWzgyNLZ3Qaaqvtjv4qzsU155WBv/9Sc9+8rP9OwrP1Nt8Kg2/ut5lVcecXtqAAAAAIB6wpZLSAYMGKANGzYoMzNTxhjl5eVp8eLFatu2rfr166ehQ4dq2LBhatKkidLT05WUlKTJkydr2rRpWrZsmQKBgHJzc9W6dWuNHj1aWVlZMsZo0qRJio6OtmPKjjq8eWnosTHWd+ObXtQlg6fZXt8YS+1/crOMseTx2H8Wxs79f5dlBU8YsyxLO/evVbdOt9peHwAAAABQ/3nMsRsTNBDFxcXndT1h2fotkqS4nj3CPaWTVH2xX/9v6X0njV+W9bSiW7Y7xVfUb+WVR7T4L2NVU3s0NNbEH6Pbhz6v2EALF2cG4GydT2/96o21J2w3G8zHYAPAMee7ZgWAxsy5myAgJLplOzVtf+0JY00Tr22Q4YUkxQZa6PouY+T3xUiS/L4YXd9lLOEFAAAAAOCsEWC45OL/ekA69ikcXp8uHvSAuxOyWdeO6aHAIjbQQl073uzyjAAAAAAA9QkBhku8/ii17HmHJKllr1/I649yeUb28np9Gpg2WZI0KO0BPkIVAAAAAHBObLmJJ85O8yuHKOY/Oinm4svdnoojLvtRZ4275UXFNW3l9lQAAAAAAPUMZ2C4rLGEF8cQXgAAAAAAzgcBBgAAAAAAiHgEGAAAAAAAIOIRYAAAAAAAgIhHgAEAAAAAACIeAQYAAAAAAIh4BBgAAAAAACDi+d2eQCQwllFczx6hxx6vx+UZAQAAAACA43EGhhQKLCq37SS8AAAAAAAgAhFgAAAAAACAiEeA8S1jjAKdr5Axxu2pAAAAAACA7yHA+JbH4znh3wAAAAAAIHIQYAAAAAAAgIhHgAEAAAAAACIeAQYAAAAAAIh4BBgAAAAAACDiEWAAAAAAAICIR4ABAAAAAAAiHgEGAAAAAACIeAQYAAAAAAAg4hFgAAAAAACAiEeAAQAAAAAAIh4BBgAAAAAAiHgEGAAAAAAAIOIRYAAAAAAAgIhHgAEAAAAAACIeAQYAAAAAAIh4BBgAAAAAACDi+d2eAACg4TGWpWaD+5405vGSmwMAAOD82BJgWJalmTNnaufOnYqKilJubq7atWsXej43N1cFBQWKjY2VJM2bN0/ffPONpk2bpmAwKGOMZs2apcTERC1evFgrVqxQQkKCJOnhhx9WYmKiHdMGAITJsaCibP0WSVJczx6EFwAAALggtgQYa9asUXV1tZYvX67CwkLNnj1b8+fPDz1fVFSkRYsWhUIJqS7UGDVqlPr376/33ntPTz75pJ555hkVFRVpzpw5Sk1NtWOqAAAAAACgHrAlwNi6dat69eolSeratau2b98ees6yLO3fv18zZszQoUOHlJGRoYyMDOXk5Cg+Pl6SFAwGFR0dLaku7Fi4cKFKS0vVp08f3X333WesXVVVpeLiYjteFgDUO8nJyWE5zvn01u/XpjcDaCjC0VtZswLAd862r9oSYJSVlSkuLi607fP5VFtbK7/fr4qKCo0aNUq33367gsGgsrOzlZqaqk6dOkmS9uzZozlz5ujZZ5+VJA0ePFhZWVmKi4vTvffeq7Vr16pv376nrCtJ0dHRYVuwAwDqhKO30psB4DusWQHg3NlyQXJcXJzKy8tD25Zlye+vy0oCgYCys7MVCAQUFxentLQ0lZSUSJI2bdqkCRMm6De/+Y0SExNljNGYMWOUkJCgqKgo3XjjjdqxY4cdUwYAAAAAABHMlgCjW7duWrdunSSpsLBQHTt2DD23b98+ZWVlKRgMqqamRgUFBUpJSdGmTZv06KOPatGiRercubOkujM5hgwZovLychljtHnzZu6FAQD1hLGM4nr2UFzPHjKWcXs6AAAAqOdsuYRkwIAB2rBhgzIzM2WMUV5enhYvXqy2bduqX79+Gjp0qIYNG6YmTZooPT1dSUlJmjx5smpqajRlyhRJUvv27TVr1ixNmjRJ2dnZioqK0nXXXacbb7zRjikDAMLM4/Wc8jEAAABwPjzGmAb1Z7Hi4mKuJwSAMKO3AkB40VcB4NzZcgkJAAAAAABAOBFgAAAAAACAiEeAAQAAAAAAIh4BBgAAAAAAiHgEGAAAAAAAIOIRYAAAAAAAgIhHgAEAAAAAACIeAQYAAAAAAIh4BBgAAAAAACDiEWAAAAAAAICIR4ABAAAAAAAint/tCYRbVVWViouL3Z4GAEQEv9+vpKSkCz4OvRUAvhOO3kpfBYDvnG1f9RhjjAPzAQAAAAAAOG9cQgIAAAAAACIeAQYAAAAAAIh4BBgAAAAAACDiEWAAAAAAAICIR4ABAAAAAAAiHgEGAAAAAACIeAQYkoLBoKZOnarMzEyNHDlSH330kWO1v/jiC914443avXu3YzVvueUWjR49WqNHj9bUqVMdqblgwQINHz5ct956q1555RVHaq5cuTL0OocNG6bOnTvr66+/trVmTU2NJk+erMzMTGVlZTn237W6ulqTJ0/WsGHDdMcdd2jfvn221vvnP/+p0aNHS5L279+vESNGKCsrSw899JAsy3Kk7jF5eXlatmyZIzWLi4uVlZWl0aNH684779ShQ4ccqbtr1y6NGDFCmZmZmjlzpoLBoG11w4W+6gyne6sbfVVyp7c63Vcld3qrG331+3Wd6q31va9K9FYnsGa1F2tW1qwXzMC8/fbbZsqUKcYYYzZt2mTuueceR+pWV1ebX/7yl2bgwIFm165djtQ8evSoSU9Pd6TWMZs2bTJ33323CQaDpqyszDz99NOO1jfGmJkzZ5qXXnrJ9jpvv/22ue+++4wxxqxfv97ce++9ttc0xpj8/Hzz4IMPGmOM2b17t7njjjtsq7Vw4UIzZMgQc9tttxljjLn77rvNpk2bjDHGTJ8+3axevdqRul988YW58847Tb9+/czSpUsdqTly5EizY8cOY4wxy5YtM3l5eY7UHT9+vPnHP/5hjDEmJyfHtu9xONFX7ed2b3WqrxrjTm91sq8a405vdaOvnqquE721IfRVY+itdnO7rxrDmjWcWLM2zDUrZ2BI6t+/vx555BFJ0ieffKJWrVo5UnfOnDnKzMzUj370I0fqSVJJSYkqKyt1xx13KDs7W4WFhbbXXL9+vTp27KgJEybonnvuUZ8+fWyvebxt27Zp165dGj58uO212rdvr2AwKMuyVFZWJr/fb3tNqS7t7N27tyQpMTHR1hS9bdu2mjt3bmi7qKhI11xzjSSpd+/e2rhxoyN1y8vLNXHiRKWnp9tS71Q1n3zySSUnJ0uq+ytYdHS0I3Xnzp2rHj16qLq6WqWlpWrZsqUtdcOJvmo/N3urk31Vcqe3OtlXJXd6qxt99VR1neitDaGvSvRWu7FmtR9rVns0pjUrAca3/H6/cnJy9Mgjj2jQoEG211u5cqUSEhLUq1cv22sdLyYmRnfeeaeee+45Pfzww3rggQdUW1tra80jR45o+/bteuqpp0I1jTG21jzeggULNGHCBEdqNW3aVAcOHNBPf/pTTZ8+/aRTx+ySnJystWvXyhijwsJCHTx40LbTYQcNGnTCLzljjDwejyQpNjZW33zzjSN127RpoyuvvNKWWqereWzhVlBQoCVLlmjs2LGO1PX5fDpw4ICGDBmiI0eOqH379rbUDTf6qr3c7K1O9lXJnd7qZF+V3OmtbvTVU9V1orc2lL4q0VvtxJrVfqxZ7dGY1qwEGMeZM2eO3nrrLU2fPl0VFRW21nr11Ve1ceNGjR49WsXFxcrJyVFpaamtNaW6tPXmm2+Wx+NR+/bt1bx5c9vrNm/eXD179lRUVJQSExMVHR2tw4cP21rzmK+//lp79uxRWlqaI/Wef/559ezZU2+99ZZee+01TZkyRVVVVbbX/fnPf664uDhlZ2dr7dq1SklJkc/ns72uJHm937WR8vJyXXTRRY7Udcubb76phx56SAsXLlRCQoJjdS+99FKtXr1aI0aM0OzZsx2re6Hoq/Zxq7c63Vcld3qrm31Vorc6ob72VYneahfWrKxZG5KGumYlwJD05z//WQsWLJAkBQIBeTwe23+QXnzxRS1ZskT5dz+wdQAABZ9JREFU+flKTk7WnDlz1Lp1a1trStKKFStC/yMdPHhQZWVltte9+uqr9d5778kYo4MHD6qyslLNmze3teYxW7Zs0fXXX+9ILUm66KKLFB8fL0lq1qyZamtrHbkx2LZt23T11VcrPz9f/fv3V5s2bWyvecxPfvITbd68WZK0bt06de/e3bHaTnvttddCP7dOfo/vueee0E2uYmNjT/gFHKnoq/bXdau3Ot1XJXd6q5t9VaK32q0+9lWJ3sqaNXxYs9JX7eBEb3XmYqcIN3DgQE2dOlUjR45UbW2tpk2bZtt1Qm7LyMjQ1KlTNWLECHk8HuXl5dl+zVvfvn21ZcsWZWRkyBijGTNmOJa07t27V5dddpkjtSRp7NixmjZtmrKyslRTU6NJkyapadOmttdt166dnnrqKf3xj39UfHy8Hn30UdtrHpOTk6Pp06frySefVGJioiOns7ohGAzq0Ucf1SWXXKKJEydKknr06KH77rvP9tp33XWXpkyZoiZNmigQCCg3N9f2mheKvmr/r1e3eqvTfVVyp7e62VcleqvdvbU+9lWJ3sqaNXxYs9JX7eBEb/UYJy/sAgAAAAAAOA/143w5AAAAAADQqBFgAAAAAACAiEeAAQAAAAAAIh4BBgAAAAAAiHgEGAAAAAAAIOIRYAA2W7ZsmebOnXvOX7d8+XLV1NRo8+bNmjRpkg0zA4D6i94KAOFHb0WkI8AAItSCBQtkWZbb0wCABoXeCgDhR2+FU/xuTwBww8qVK7V27VodPXpUpaWlys7O1jvvvKMPP/xQv/rVr/TZZ59p9erVqq2tVXx8vObOnatXXnlFBQUFeuKJJ5STk6MuXbpo5MiRpzz+Bx98oLy8PDVr1kxer1ddu3aVJOXn5+v111+Xx+PRTTfdpOzsbE2ZMkXGGH366aeqqKjQnDlzVFBQoNLSUk2aNEljxozR/v379Ytf/EKHDx9W3759NXHiRCe/XQBwVuitABB+9FbgO5yBgUarvLxcf/jDHzRu3DgtW7ZMzzzzjGbNmqUVK1boyy+/1PPPP6+lS5eqtrZW27Zt08iRI1VZWakpU6aopqbmtL8EJOmxxx7TE088ocWLF+uyyy6TJO3atUtvvvmmli5dqqVLl2rNmjXas2ePJKlNmzZ64YUXNHHiRD3++OO67bbb1Lp1a/3ud7+TJFVVVWnevHl68cUXtWTJEvu/OQBwnuitABB+9FagDmdgoNFKTk6WJMXHx6tDhw7yeDxq1qyZampq1KRJE91///1q2rSpPvvsM9XW1kqS7rrrLg0fPlwrV64847EPHjyo9u3bS5K6deumjz76SP/+97/1ySefaOzYsZKkr776Sh999JEkKS0tTZJ01VVXKS8v76TjJSUlKSoqSpLk9/NjCyBy0VsBIPzorUAdzsBAo+XxeE45XlNTozVr1uj3v/+9pk+fLsuyZIxRdXW18vLyNGvWLM2cOVPV1dWnPXbr1q21e/duSdK2bdskSYmJibr88sv1wgsvKD8/X7feeqs6duwoSSoqKpIkFRQUKCkpKTS/Y9cSnm6uABBp6K0AEH70VqAOkRjwPX6/X4FAQLfeequioqLUunVrff755/rtb3+rPn36aPjw4fr888/1xBNPaOrUqac8xuOPP66cnBzFxsYqNjZWzZo1U6dOnXTddddpxIgRqq6uVpcuXXTxxRdLktatW6d33nlHlmXpsccekyR1795dd911lyZMmODYawcAu9BbASD86K1obDzGGOP2JIDGbMqUKbrpppvUu3dvt6cCAA0GvRUAwo/eCrdxBgZwnj755BPl5OScNN6jRw/dd999LswIAOo/eisAhB+9FQ0FZ2AAAAAAAICIx008AQAAAABAxCPAAAAAAAAAEY8AAwAAAAAARDwCDAAAAAAAEPEIMAAAAAAAQMT7/1ziq97yKve8AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1080x360 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.catplot(x='max_depth', y='valid', col='model_type', data=lgb_xgb_combined, kind='boxen');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hyperparameter Impact Evaluation using Linear Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Instead of a DecisionTreeRegressor as shown previously, we can also use linear regression to evaluate the statistical significance of different features concerning the validation AUC score. \n",
    "\n",
    "For the LightGBM Dummy model, where the regression explains 68% of the variation in outcomes, we find that only the min_gain_to_split regularization parameter was not significant, as shown below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-24T18:03:49.549666Z",
     "start_time": "2018-10-24T18:03:45.345614Z"
    },
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " Lightgbm Dummies\n",
      "                            OLS Regression Results                            \n",
      "==============================================================================\n",
      "Dep. Variable:                  valid   R-squared:                       0.687\n",
      "Model:                            OLS   Adj. R-squared:                  0.673\n",
      "Method:                 Least Squares   F-statistic:                     26.94\n",
      "Date:                Sat, 20 Apr 2019   Prob (F-statistic):           7.92e-55\n",
      "Time:                        14:39:20   Log-Likelihood:                 1018.7\n",
      "No. Observations:                 396   AIC:                            -2001.\n",
      "Df Residuals:                     378   BIC:                            -1930.\n",
      "Df Model:                          17                                         \n",
      "Covariance Type:                  HC3                                         \n",
      "========================================================================================\n",
      "                           coef    std err          z      P>|z|      [0.025      0.975]\n",
      "----------------------------------------------------------------------------------------\n",
      "const                    0.6145      0.005    127.970      0.000       0.605       0.624\n",
      "boosting_gbtree          0.0056      0.002      2.866      0.004       0.002       0.009\n",
      "learning_rate_0.1        0.0501      0.003     18.977      0.000       0.045       0.055\n",
      "learning_rate_0.3        0.0516      0.003     19.150      0.000       0.046       0.057\n",
      "max_depth_4              0.0060      0.005      1.094      0.274      -0.005       0.017\n",
      "max_depth_5              0.0096      0.005      1.823      0.068      -0.001       0.020\n",
      "max_depth_6              0.0153      0.005      3.024      0.002       0.005       0.025\n",
      "max_depth_7              0.0194      0.005      3.753      0.000       0.009       0.030\n",
      "max_depth_8              0.0196      0.005      3.733      0.000       0.009       0.030\n",
      "max_depth_9              0.0266      0.005      5.176      0.000       0.017       0.037\n",
      "max_depth_10             0.0307      0.005      5.954      0.000       0.021       0.041\n",
      "max_depth_11             0.0285      0.005      5.484      0.000       0.018       0.039\n",
      "max_depth_12             0.0312      0.005      6.178      0.000       0.021       0.041\n",
      "max_depth_13             0.0320      0.005      6.218      0.000       0.022       0.042\n",
      "colsample_bytree_0.8    -0.0112      0.003     -4.143      0.000      -0.017      -0.006\n",
      "colsample_bytree_1.0    -0.0278      0.003     -8.388      0.000      -0.034      -0.021\n",
      "min_gain_to_split_1     -0.0009      0.003     -0.307      0.759      -0.006       0.005\n",
      "min_gain_to_split_5     -0.0016      0.002     -0.726      0.468      -0.006       0.003\n",
      "==============================================================================\n",
      "Omnibus:                       11.763   Durbin-Watson:                   0.856\n",
      "Prob(Omnibus):                  0.003   Jarque-Bera (JB):               11.104\n",
      "Skew:                          -0.361   Prob(JB):                      0.00388\n",
      "Kurtosis:                       2.609   Cond. No.                         17.1\n",
      "==============================================================================\n",
      "\n",
      "Warnings:\n",
      "[1] Standard Errors are heteroscedasticity robust (HC3)\n",
      "\n",
      " Lightgbm Factors\n",
      "                            OLS Regression Results                            \n",
      "==============================================================================\n",
      "Dep. Variable:                  valid   R-squared:                       0.508\n",
      "Model:                            OLS   Adj. R-squared:                  0.486\n",
      "Method:                 Least Squares   F-statistic:                     23.49\n",
      "Date:                Sat, 20 Apr 2019   Prob (F-statistic):           4.59e-49\n",
      "Time:                        14:39:20   Log-Likelihood:                 1080.3\n",
      "No. Observations:                 396   AIC:                            -2125.\n",
      "Df Residuals:                     378   BIC:                            -2053.\n",
      "Df Model:                          17                                         \n",
      "Covariance Type:                  HC3                                         \n",
      "========================================================================================\n",
      "                           coef    std err          z      P>|z|      [0.025      0.975]\n",
      "----------------------------------------------------------------------------------------\n",
      "const                    0.6463      0.007     93.983      0.000       0.633       0.660\n",
      "boosting_gbdt            0.0044      0.002      2.630      0.009       0.001       0.008\n",
      "learning_rate_0.1        0.0228      0.002     10.055      0.000       0.018       0.027\n",
      "learning_rate_0.3        0.0237      0.002     10.129      0.000       0.019       0.028\n",
      "max_depth_4              0.0074      0.007      1.088      0.277      -0.006       0.021\n",
      "max_depth_5              0.0074      0.007      1.035      0.301      -0.007       0.021\n",
      "max_depth_6              0.0110      0.007      1.622      0.105      -0.002       0.024\n",
      "max_depth_7              0.0114      0.007      1.693      0.090      -0.002       0.025\n",
      "max_depth_8              0.0100      0.007      1.475      0.140      -0.003       0.023\n",
      "max_depth_9              0.0107      0.007      1.487      0.137      -0.003       0.025\n",
      "max_depth_10             0.0114      0.007      1.590      0.112      -0.003       0.026\n",
      "max_depth_11             0.0082      0.007      1.137      0.255      -0.006       0.022\n",
      "max_depth_12             0.0066      0.007      0.914      0.361      -0.008       0.021\n",
      "max_depth_13             0.0051      0.007      0.696      0.486      -0.009       0.019\n",
      "colsample_bytree_0.8    -0.0041      0.002     -1.960      0.050      -0.008   -4.83e-07\n",
      "colsample_bytree_1.0    -0.0251      0.003     -8.210      0.000      -0.031      -0.019\n",
      "min_gain_to_split_1     -0.0007      0.002     -0.357      0.721      -0.005       0.003\n",
      "min_gain_to_split_5      0.0017      0.002      0.911      0.362      -0.002       0.005\n",
      "==============================================================================\n",
      "Omnibus:                      250.569   Durbin-Watson:                   0.753\n",
      "Prob(Omnibus):                  0.000   Jarque-Bera (JB):             3487.662\n",
      "Skew:                          -2.441   Prob(JB):                         0.00\n",
      "Kurtosis:                      16.694   Cond. No.                         17.1\n",
      "==============================================================================\n",
      "\n",
      "Warnings:\n",
      "[1] Standard Errors are heteroscedasticity robust (HC3)\n",
      "\n",
      " Xgb Dummies\n",
      "                            OLS Regression Results                            \n",
      "==============================================================================\n",
      "Dep. Variable:                  valid   R-squared:                       0.776\n",
      "Model:                            OLS   Adj. R-squared:                  0.764\n",
      "Method:                 Least Squares   F-statistic:                     48.42\n",
      "Date:                Sat, 20 Apr 2019   Prob (F-statistic):           7.44e-51\n",
      "Time:                        14:39:20   Log-Likelihood:                 578.58\n",
      "No. Observations:                 216   AIC:                            -1133.\n",
      "Df Residuals:                     204   BIC:                            -1093.\n",
      "Df Model:                          11                                         \n",
      "Covariance Type:                  HC3                                         \n",
      "========================================================================================\n",
      "                           coef    std err          z      P>|z|      [0.025      0.975]\n",
      "----------------------------------------------------------------------------------------\n",
      "const                    0.6026      0.005    131.844      0.000       0.594       0.612\n",
      "boosting_gbtree          0.0003      0.002      0.129      0.897      -0.004       0.005\n",
      "learning_rate_0.1        0.0610      0.003     19.640      0.000       0.055       0.067\n",
      "learning_rate_0.3        0.0589      0.003     17.349      0.000       0.052       0.066\n",
      "max_depth_5              0.0038      0.005      0.798      0.425      -0.006       0.013\n",
      "max_depth_7              0.0161      0.004      4.004      0.000       0.008       0.024\n",
      "max_depth_9              0.0251      0.004      5.803      0.000       0.017       0.034\n",
      "max_depth_11             0.0248      0.004      5.764      0.000       0.016       0.033\n",
      "max_depth_13             0.0261      0.004      5.867      0.000       0.017       0.035\n",
      "colsample_bytree_1.0    -0.0135      0.002     -5.658      0.000      -0.018      -0.009\n",
      "min_gain_to_split_1    6.87e-16      0.003   2.34e-13      1.000      -0.006       0.006\n",
      "min_gain_to_split_5  -9.593e-16      0.003  -3.27e-13      1.000      -0.006       0.006\n",
      "==============================================================================\n",
      "Omnibus:                        7.507   Durbin-Watson:                   0.373\n",
      "Prob(Omnibus):                  0.023   Jarque-Bera (JB):                7.659\n",
      "Skew:                           0.434   Prob(JB):                       0.0217\n",
      "Kurtosis:                       2.688   Cond. No.                         9.58\n",
      "==============================================================================\n",
      "\n",
      "Warnings:\n",
      "[1] Standard Errors are heteroscedasticity robust (HC3)\n"
     ]
    }
   ],
   "source": [
    "features = ['boosting','learning_rate', 'max_depth', 'colsample_bytree', 'min_gain_to_split']\n",
    "path = Path('ols_results')\n",
    "for model, data in lgb_xgb_combined.groupby('model_type'):\n",
    "    y = data.valid \n",
    "    X = pd.get_dummies(data[features].dropna(how='all', axis=1), columns=features, drop_first=True)\n",
    "    ols = OLS(endog=y, exog=add_constant(X)).fit(cov_type='HC3')\n",
    "    print('\\n', model)\n",
    "    print(ols.summary())    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### CatBoost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-10T02:10:43.464065Z",
     "start_time": "2018-10-10T02:10:43.433313Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 96 entries, 0 to 95\n",
      "Data columns (total 9 columns):\n",
      "rounds                96 non-null float64\n",
      "train                 96 non-null float64\n",
      "valid                 96 non-null float64\n",
      "max_depth             96 non-null int64\n",
      "max_bin               96 non-null int64\n",
      "max_ctr_complexity    96 non-null int64\n",
      "one_hot_max_size      48 non-null float64\n",
      "random_strength       48 non-null float64\n",
      "time                  96 non-null float64\n",
      "dtypes: float64(6), int64(3)\n",
      "memory usage: 7.5 KB\n"
     ]
    }
   ],
   "source": [
    "cat_features = ['max_depth', 'max_bin', 'max_ctr_complexity',  'one_hot_max_size',  'random_strength']\n",
    "with pd.HDFStore('results.h5') as store:\n",
    "    cat_data = store['catboost/factors']\n",
    "cat_data.info()    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-10T02:12:12.738786Z",
     "start_time": "2018-10-10T02:12:12.719527Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeRegressor(criterion='mse', max_depth=4, max_features=None,\n",
       "           max_leaf_nodes=None, min_impurity_decrease=0.0,\n",
       "           min_impurity_split=None, min_samples_leaf=10,\n",
       "           min_samples_split=5, min_weight_fraction_leaf=0.0,\n",
       "           presort=False, random_state=42, splitter='best')"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reg_tree.fit(X=cat_data[cat_features].fillna(-1), y=cat_data.valid)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-10-10T02:12:34.982118Z",
     "start_time": "2018-10-10T02:12:34.920447Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.40.1 (20161225.0304)\n",
       " -->\n",
       "<!-- Title: Tree Pages: 1 -->\n",
       "<svg width=\"692pt\" height=\"477pt\"\n",
       " viewBox=\"0.00 0.00 692.00 477.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 473)\">\n",
       "<title>Tree</title>\n",
       "<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-4,4 -4,-473 688,-473 688,4 -4,4\"/>\n",
       "<!-- 0 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>0</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.560784\" stroke=\"#000000\" d=\"M281.5,-469C281.5,-469 186.5,-469 186.5,-469 180.5,-469 174.5,-463 174.5,-457 174.5,-457 174.5,-413 174.5,-413 174.5,-407 180.5,-401 186.5,-401 186.5,-401 281.5,-401 281.5,-401 287.5,-401 293.5,-407 293.5,-413 293.5,-413 293.5,-457 293.5,-457 293.5,-463 287.5,-469 281.5,-469\"/>\n",
       "<text text-anchor=\"start\" x=\"182.5\" y=\"-453.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">max_depth ≤ 4.5</text>\n",
       "<text text-anchor=\"start\" x=\"196\" y=\"-438.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.005</text>\n",
       "<text text-anchor=\"start\" x=\"193\" y=\"-423.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 96</text>\n",
       "<text text-anchor=\"start\" x=\"193\" y=\"-408.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.606</text>\n",
       "</g>\n",
       "<!-- 1 -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>1</title>\n",
       "<path fill=\"transparent\" stroke=\"#000000\" d=\"M198,-365C198,-365 112,-365 112,-365 106,-365 100,-359 100,-353 100,-353 100,-309 100,-309 100,-303 106,-297 112,-297 112,-297 198,-297 198,-297 204,-297 210,-303 210,-309 210,-309 210,-353 210,-353 210,-359 204,-365 198,-365\"/>\n",
       "<text text-anchor=\"start\" x=\"108\" y=\"-349.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">max_bin ≤ 80.0</text>\n",
       "<text text-anchor=\"start\" x=\"117\" y=\"-334.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.001</text>\n",
       "<text text-anchor=\"start\" x=\"114\" y=\"-319.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 24</text>\n",
       "<text text-anchor=\"start\" x=\"114\" y=\"-304.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.515</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;1 -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>0&#45;&gt;1</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M208.1324,-400.9465C201.3804,-392.0578 194.0226,-382.3716 186.9941,-373.1188\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"189.7617,-370.976 180.9257,-365.13 184.1875,-375.2102 189.7617,-370.976\"/>\n",
       "<text text-anchor=\"middle\" x=\"177.5418\" y=\"-386.1999\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">True</text>\n",
       "</g>\n",
       "<!-- 4 -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>4</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.749020\" stroke=\"#000000\" d=\"M388,-365C388,-365 240,-365 240,-365 234,-365 228,-359 228,-353 228,-353 228,-309 228,-309 228,-303 234,-297 240,-297 240,-297 388,-297 388,-297 394,-297 400,-303 400,-309 400,-309 400,-353 400,-353 400,-359 394,-365 388,-365\"/>\n",
       "<text text-anchor=\"start\" x=\"236\" y=\"-349.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">max_ctr_complexity ≤ 1.5</text>\n",
       "<text text-anchor=\"start\" x=\"276\" y=\"-334.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.003</text>\n",
       "<text text-anchor=\"start\" x=\"273\" y=\"-319.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 72</text>\n",
       "<text text-anchor=\"start\" x=\"273\" y=\"-304.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.637</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;4 -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>0&#45;&gt;4</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M260.195,-400.9465C267.0325,-392.0578 274.4834,-382.3716 281.601,-373.1188\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"284.4232,-375.1903 287.7462,-365.13 278.8748,-370.9223 284.4232,-375.1903\"/>\n",
       "<text text-anchor=\"middle\" x=\"290.9796\" y=\"-386.22\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">False</text>\n",
       "</g>\n",
       "<!-- 2 -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>2</title>\n",
       "<path fill=\"transparent\" stroke=\"#000000\" d=\"M86,-253.5C86,-253.5 12,-253.5 12,-253.5 6,-253.5 0,-247.5 0,-241.5 0,-241.5 0,-212.5 0,-212.5 0,-206.5 6,-200.5 12,-200.5 12,-200.5 86,-200.5 86,-200.5 92,-200.5 98,-206.5 98,-212.5 98,-212.5 98,-241.5 98,-241.5 98,-247.5 92,-253.5 86,-253.5\"/>\n",
       "<text text-anchor=\"start\" x=\"18.5\" y=\"-238.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"8\" y=\"-223.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 12</text>\n",
       "<text text-anchor=\"start\" x=\"8\" y=\"-208.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.515</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;2 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>1&#45;&gt;2</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M120.2916,-296.9465C108.3913,-285.2707 95.0886,-272.219 83.2267,-260.5809\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"85.6419,-258.0473 76.0526,-253.5422 80.7395,-263.0439 85.6419,-258.0473\"/>\n",
       "</g>\n",
       "<!-- 3 -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>3</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.003922\" stroke=\"#000000\" d=\"M202,-253.5C202,-253.5 128,-253.5 128,-253.5 122,-253.5 116,-247.5 116,-241.5 116,-241.5 116,-212.5 116,-212.5 116,-206.5 122,-200.5 128,-200.5 128,-200.5 202,-200.5 202,-200.5 208,-200.5 214,-206.5 214,-212.5 214,-212.5 214,-241.5 214,-241.5 214,-247.5 208,-253.5 202,-253.5\"/>\n",
       "<text text-anchor=\"start\" x=\"127\" y=\"-238.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.001</text>\n",
       "<text text-anchor=\"start\" x=\"124\" y=\"-223.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 12</text>\n",
       "<text text-anchor=\"start\" x=\"124\" y=\"-208.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.515</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;3 -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>1&#45;&gt;3</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M158.2744,-296.9465C159.3017,-286.2621 160.4399,-274.4254 161.4832,-263.5742\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"164.9746,-263.8313 162.4479,-253.5422 158.0067,-263.1612 164.9746,-263.8313\"/>\n",
       "</g>\n",
       "<!-- 5 -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>5</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.439216\" stroke=\"#000000\" d=\"M348,-261C348,-261 262,-261 262,-261 256,-261 250,-255 250,-249 250,-249 250,-205 250,-205 250,-199 256,-193 262,-193 262,-193 348,-193 348,-193 354,-193 360,-199 360,-205 360,-205 360,-249 360,-249 360,-255 354,-261 348,-261\"/>\n",
       "<text text-anchor=\"start\" x=\"258\" y=\"-245.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">max_bin ≤ 80.0</text>\n",
       "<text text-anchor=\"start\" x=\"267\" y=\"-230.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.003</text>\n",
       "<text text-anchor=\"start\" x=\"264\" y=\"-215.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 24</text>\n",
       "<text text-anchor=\"start\" x=\"264\" y=\"-200.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.586</text>\n",
       "</g>\n",
       "<!-- 4&#45;&gt;5 -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>4&#45;&gt;5</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M311.0531,-296.9465C310.346,-288.776 309.5806,-279.9318 308.8397,-271.3697\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"312.3028,-270.791 307.9536,-261.13 305.3288,-271.3945 312.3028,-270.791\"/>\n",
       "</g>\n",
       "<!-- 8 -->\n",
       "<g id=\"node9\" class=\"node\">\n",
       "<title>8</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.905882\" stroke=\"#000000\" d=\"M485.5,-261C485.5,-261 390.5,-261 390.5,-261 384.5,-261 378.5,-255 378.5,-249 378.5,-249 378.5,-205 378.5,-205 378.5,-199 384.5,-193 390.5,-193 390.5,-193 485.5,-193 485.5,-193 491.5,-193 497.5,-199 497.5,-205 497.5,-205 497.5,-249 497.5,-249 497.5,-255 491.5,-261 485.5,-261\"/>\n",
       "<text text-anchor=\"start\" x=\"386.5\" y=\"-245.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">max_depth ≤ 7.5</text>\n",
       "<text text-anchor=\"start\" x=\"400\" y=\"-230.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.001</text>\n",
       "<text text-anchor=\"start\" x=\"397\" y=\"-215.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 48</text>\n",
       "<text text-anchor=\"start\" x=\"397\" y=\"-200.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.662</text>\n",
       "</g>\n",
       "<!-- 4&#45;&gt;8 -->\n",
       "<g id=\"edge8\" class=\"edge\">\n",
       "<title>4&#45;&gt;8</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M354.6022,-296.9465C365.7356,-287.6088 377.9183,-277.3911 389.4489,-267.7203\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"391.8938,-270.2378 397.3065,-261.13 387.3954,-264.8745 391.8938,-270.2378\"/>\n",
       "</g>\n",
       "<!-- 6 -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>6</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.329412\" stroke=\"#000000\" d=\"M230,-149.5C230,-149.5 156,-149.5 156,-149.5 150,-149.5 144,-143.5 144,-137.5 144,-137.5 144,-108.5 144,-108.5 144,-102.5 150,-96.5 156,-96.5 156,-96.5 230,-96.5 230,-96.5 236,-96.5 242,-102.5 242,-108.5 242,-108.5 242,-137.5 242,-137.5 242,-143.5 236,-149.5 230,-149.5\"/>\n",
       "<text text-anchor=\"start\" x=\"155\" y=\"-134.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.003</text>\n",
       "<text text-anchor=\"start\" x=\"152\" y=\"-119.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 12</text>\n",
       "<text text-anchor=\"start\" x=\"152\" y=\"-104.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.569</text>\n",
       "</g>\n",
       "<!-- 5&#45;&gt;6 -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>5&#45;&gt;6</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M268.327,-192.9465C255.7531,-181.2707 241.6974,-168.219 229.1641,-156.5809\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"231.2934,-153.7819 221.5839,-149.5422 226.5302,-158.9115 231.2934,-153.7819\"/>\n",
       "</g>\n",
       "<!-- 7 -->\n",
       "<g id=\"node8\" class=\"node\">\n",
       "<title>7</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.549020\" stroke=\"#000000\" d=\"M346,-149.5C346,-149.5 272,-149.5 272,-149.5 266,-149.5 260,-143.5 260,-137.5 260,-137.5 260,-108.5 260,-108.5 260,-102.5 266,-96.5 272,-96.5 272,-96.5 346,-96.5 346,-96.5 352,-96.5 358,-102.5 358,-108.5 358,-108.5 358,-137.5 358,-137.5 358,-143.5 352,-149.5 346,-149.5\"/>\n",
       "<text text-anchor=\"start\" x=\"271\" y=\"-134.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.002</text>\n",
       "<text text-anchor=\"start\" x=\"268\" y=\"-119.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 12</text>\n",
       "<text text-anchor=\"start\" x=\"268\" y=\"-104.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.604</text>\n",
       "</g>\n",
       "<!-- 5&#45;&gt;7 -->\n",
       "<g id=\"edge7\" class=\"edge\">\n",
       "<title>5&#45;&gt;7</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M306.3097,-192.9465C306.7207,-182.2621 307.1759,-170.4254 307.5933,-159.5742\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"311.0921,-159.6693 307.9791,-149.5422 304.0973,-159.4002 311.0921,-159.6693\"/>\n",
       "</g>\n",
       "<!-- 9 -->\n",
       "<g id=\"node10\" class=\"node\">\n",
       "<title>9</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.768627\" stroke=\"#000000\" d=\"M470,-149.5C470,-149.5 396,-149.5 396,-149.5 390,-149.5 384,-143.5 384,-137.5 384,-137.5 384,-108.5 384,-108.5 384,-102.5 390,-96.5 396,-96.5 396,-96.5 470,-96.5 470,-96.5 476,-96.5 482,-102.5 482,-108.5 482,-108.5 482,-137.5 482,-137.5 482,-143.5 476,-149.5 470,-149.5\"/>\n",
       "<text text-anchor=\"start\" x=\"395\" y=\"-134.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.002</text>\n",
       "<text text-anchor=\"start\" x=\"392\" y=\"-119.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 16</text>\n",
       "<text text-anchor=\"start\" x=\"395.5\" y=\"-104.3\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.64</text>\n",
       "</g>\n",
       "<!-- 8&#45;&gt;9 -->\n",
       "<g id=\"edge9\" class=\"edge\">\n",
       "<title>8&#45;&gt;9</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M436.3628,-192.9465C435.8491,-182.2621 435.2801,-170.4254 434.7584,-159.5742\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"438.2523,-159.3625 434.2761,-149.5422 431.2604,-159.6987 438.2523,-159.3625\"/>\n",
       "</g>\n",
       "<!-- 10 -->\n",
       "<g id=\"node11\" class=\"node\">\n",
       "<title>10</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.972549\" stroke=\"#000000\" d=\"M642,-157C642,-157 512,-157 512,-157 506,-157 500,-151 500,-145 500,-145 500,-101 500,-101 500,-95 506,-89 512,-89 512,-89 642,-89 642,-89 648,-89 654,-95 654,-101 654,-101 654,-145 654,-145 654,-151 648,-157 642,-157\"/>\n",
       "<text text-anchor=\"start\" x=\"508\" y=\"-141.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">random_strength ≤ 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"546.5\" y=\"-126.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"536\" y=\"-111.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 32</text>\n",
       "<text text-anchor=\"start\" x=\"536\" y=\"-96.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.673</text>\n",
       "</g>\n",
       "<!-- 8&#45;&gt;10 -->\n",
       "<g id=\"edge10\" class=\"edge\">\n",
       "<title>8&#45;&gt;10</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M483.5138,-192.9465C496.234,-183.4293 510.176,-172.9978 523.3206,-163.163\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"525.4738,-165.9232 531.3839,-157.13 521.2802,-160.3184 525.4738,-165.9232\"/>\n",
       "</g>\n",
       "<!-- 11 -->\n",
       "<g id=\"node12\" class=\"node\">\n",
       "<title>11</title>\n",
       "<path fill=\"#e58139\" fill-opacity=\"0.945098\" stroke=\"#000000\" d=\"M556,-53C556,-53 482,-53 482,-53 476,-53 470,-47 470,-41 470,-41 470,-12 470,-12 470,-6 476,0 482,0 482,0 556,0 556,0 562,0 568,-6 568,-12 568,-12 568,-41 568,-41 568,-47 562,-53 556,-53\"/>\n",
       "<text text-anchor=\"start\" x=\"488.5\" y=\"-37.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"478\" y=\"-22.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 16</text>\n",
       "<text text-anchor=\"start\" x=\"478\" y=\"-7.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.668</text>\n",
       "</g>\n",
       "<!-- 10&#45;&gt;11 -->\n",
       "<g id=\"edge11\" class=\"edge\">\n",
       "<title>10&#45;&gt;11</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M556.5514,-88.9777C551.2678,-80.187 545.5684,-70.7044 540.2452,-61.8477\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"543.2282,-60.0165 535.0768,-53.2485 537.2285,-63.6225 543.2282,-60.0165\"/>\n",
       "</g>\n",
       "<!-- 12 -->\n",
       "<g id=\"node13\" class=\"node\">\n",
       "<title>12</title>\n",
       "<path fill=\"#e58139\" stroke=\"#000000\" d=\"M672,-53C672,-53 598,-53 598,-53 592,-53 586,-47 586,-41 586,-41 586,-12 586,-12 586,-6 592,0 598,0 598,0 672,0 672,0 678,0 684,-6 684,-12 684,-12 684,-41 684,-41 684,-47 678,-53 672,-53\"/>\n",
       "<text text-anchor=\"start\" x=\"604.5\" y=\"-37.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mse = 0.0</text>\n",
       "<text text-anchor=\"start\" x=\"594\" y=\"-22.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">samples = 16</text>\n",
       "<text text-anchor=\"start\" x=\"594\" y=\"-7.8\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">value = 0.677</text>\n",
       "</g>\n",
       "<!-- 10&#45;&gt;12 -->\n",
       "<g id=\"edge12\" class=\"edge\">\n",
       "<title>10&#45;&gt;12</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M597.4486,-88.9777C602.7322,-80.187 608.4316,-70.7044 613.7548,-61.8477\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"616.7715,-63.6225 618.9232,-53.2485 610.7718,-60.0165 616.7715,-63.6225\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.files.Source at 0x7f0d840ec080>"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "out_file = 'results/cat_tree.dot'\n",
    "dot_data = export_graphviz(reg_tree,\n",
    "                          out_file=out_file,\n",
    "                          feature_names=cat_features,\n",
    "                          max_depth=4,\n",
    "                          filled=True,\n",
    "                          rounded=True,\n",
    "                          special_characters=True)\n",
    "if out_file is not None:\n",
    "    dot_data = Path(out_file).read_text()\n",
    "\n",
    "graphviz.Source(dot_data)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": true,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "309.112px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
